Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add alembic scripts for indices and composite keys. #76

Merged
merged 2 commits into from Feb 27, 2015
Merged

Add alembic scripts for indices and composite keys. #76

merged 2 commits into from Feb 27, 2015

Conversation

ralphbean
Copy link
Contributor

No description provided.

@ralphbean
Copy link
Contributor Author

Here's a playbook to do the eventual upgrade (with --limit staging first, then production later). https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/playbooks/manual/upgrade/datanommer.yml

@pypingou
Copy link
Member

👍 for me

@pypingou
Copy link
Member

For ansible, is there a way to monitor its progress? I see the delegate_to that should fix the potential timeout question, but then we also do not know how far it is.

@ralphbean
Copy link
Contributor Author

For ansible, is there a way to monitor its progress

Yes, I have an example handy in the rebuild-fedora-packages playbook that I can re-use for this. Will post it in a moment.

@ralphbean
Copy link
Contributor Author

This should do it:

diff --git a/playbooks/manual/upgrade/datanommer.yml b/playbooks/manual/upgrade/datanommer.yml
index 0b1037a..56a8de6 100644
--- a/playbooks/manual/upgrade/datanommer.yml
+++ b/playbooks/manual/upgrade/datanommer.yml
@@ -110,6 +110,8 @@
     command: /usr/bin/alembic upgrade head
     args:
       chdir: /usr/share/datanommer.models/
+    async: 7200
+    poll: 60

   - name: And... start the backend again
     service: name="fedmsg-hub" state=started
diff --git a/playbooks/manual/upgrade/fmn.yml b/playbooks/manual/upgrade/fmn.yml
index e07e9d0..5cc4feb 100644
--- a/playbooks/manual/upgrade/fmn.yml
+++ b/playbooks/manual/upgrade/fmn.yml
@@ -65,10 +65,14 @@
   tasks:
   - name: Stop the notification backend
     service: name="fedmsg-hub" state=stopped
+
   - name: Upgrade the database
     command: /usr/bin/alembic upgrade head
     args:
       chdir: /usr/share/fmn.lib/
+    async: 7200
+    poll: 60
+
   - name: And... start the backend again
     service: name="fedmsg-hub" state=started

@pypingou
Copy link
Member

🎆

@ralphbean
Copy link
Contributor Author

Correction: we won't actually get the stdout report of how its progressing until its done: See ansible/ansible#3887.

@ralphbean
Copy link
Contributor Author

Logs from the dev server on how long it takes to run:

$ time alembic upgrade head
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade 143ec484f5ba -> 999b2d10b4e, Add topic index
INFO  [alembic.migration] Finished in 2456.01s
INFO  [alembic.migration] Running upgrade 999b2d10b4e -> 5a167589eb8e, Add category index
INFO  [alembic.migration] Finished in 661.62s
INFO  [alembic.migration] Running upgrade 5a167589eb8e -> 19bb834d6f9, Add users composite pkey
INFO  [alembic.migration] Running 'DELETE FROM user_messages WHERE username is NULL'
INFO  [alembic.migration] Running 'DELETE FROM user_messages WHERE msg is NULL'
INFO  [alembic.migration] Creating composite primary key
INFO  [alembic.migration] Finished in 665.28s
INFO  [alembic.migration] Running upgrade 19bb834d6f9 -> 1b786d5fc66, Add packages composite pkey
INFO  [alembic.migration] Running 'DELETE FROM package_messages WHERE package is NULL'
INFO  [alembic.migration] Running 'DELETE FROM package_messages WHERE msg is NULL'
INFO  [alembic.migration] Creating composite primary key
INFO  [alembic.migration] Finished in 438.06s

real    70m21.880s
user    0m0.470s
sys     0m0.098s

ralphbean added a commit that referenced this pull request Feb 27, 2015
@ralphbean
Merge pull request #76 from fedora-infra/feature/indices
ad602c0 
Add alembic scripts for indices and composite keys.
@ralphbean ralphbean merged commit ad602c0 into develop Feb 27, 2015
@ralphbean ralphbean deleted the feature/indices branch February 27, 2015 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ralphbean @pypingou