Messages of the topic wiki.upload.complete typically come across with an invalid signature. We need to diagnose why this is and fix it. This is a tracking ticket to make sure we don't forget about this.
In the beginning, I had trouble with the wiki because the X509 cert signing had to be implemented twice, once in python and again in php. One early source of issues was that php encoded JSON with its keys in an inconsistent way which produced inconsistent message signatures. That could be the issue here again.
It could also be that the fedmsg routing_policy is set up incorrectly.
Just a note that this sometimes, very rarely, also occurs for messages of type wiki.article.edit.
That could be due to one of the app servers being different from the others in some way. 32bit vs 64bit? (just a guess at this point).
puiterwijk and I worked on some tests (here https://gist.github.com/ralphbean/5905893 and here https://gist.github.com/ralphbean/5906151) to try and track this down.
Hm, by checking the certificate CN of the messages that were invalid we found that there was no difference between hosts. For example, some messages from app02 were invalid, while others from app02 were valid.
Update the php timestamps from a float to an int after the freeze is up </note-to-self>
(testing the trac+fedmsg plugin)
After almost a year, I figured it out!
The difference is in the Summary. If it includes an escaped '"' character, php escapes that one way when it computes the signature and python escapes it another way. This produces two different signatures when the messages is signed and then later validated. They don't match, and so fedmsg complains.
We could fight with php to make that escaping work correctly on the wiki side.. but I think it will be easier and cause no damage to just remove the 'summary' field from wiki fedmsg messages alltogether. They are not used anywhere that I know of (but I'll double check).
Ok, I removed the summary in puppet commit 1da48563828a7c1bd83fb37bb2ec67ccc93ca63a.
If someone complains, we can revert that and try a more complicated solution.
Ralph - do you have an example of the difference between how PHP escapes the quote and how Python does it? What does Python do differently from here? I might try and come up with a fix so we can add summary back, if it's not something crazy.
I'm afraid I don't have an example at hand. The best I have to go on are those two example messages from the datagrepper history.
You could copy those blocks and serialize each one once in python and again in php and study the difference.
OK - I got the wiki upload messages "fixed" by removing portions of the message that get encoded differently. We don't use them anywhere; I feel comfortable removing them.
@codeblock, Here's an example of a message that gets encoded differently.
First here's the PHP version
{{{ {"i":1,"msg":{"description":"is this how I rock and roll?","file_exists":true,"major_mime":"image","media_type":"BITMAP","mime":"image/jpeg","minor_mime":"jpeg","size":608111,"title":{"mTextform":"Rock-and-roll.jpg","mUrlform":"Rock-and-roll.jpg","mDbkeyform":"Rock-and-roll.jpg","mUserCaseDBKey":"Rock-and-roll.jpg","mNamespace":6,"mInterwiki":"","mFragment":"","mArticleID":46687,"mLatestID":false,"mRestrictions":[],"mOldRestrictions":false,"mCascadeRestriction":null,"mCascadingRestrictions":[],"mRestrictionsExpiry":{"create":"infinity"},"mHasCascadingRestrictions":null,"mCascadeSources":[],"mRestrictionsLoaded":false,"mPrefixedText":"File:Rock-and-roll.jpg","mTitleProtection":false,"mDefaultNamespace":0,"mWatched":null,"mLength":-1,"mRedirect":null,"mNotificationTimestamp":[],"mBacklinkCache":{}},"url":"/w/uploads/2/2b/Rock-and-roll.jpg","user_id":8306,"user_text":"Ralph"},"msg_id":"2014-95c29490-5e21-48c6-a4c4-c0c5f7aa0bd4","timestamp":1416001297,"topic":"org.fedoraproject.stg.wiki.upload.complete","username":"apache"} }}}
Second here's the python version:
{{{ {"i":1,"msg":{"description":"is this how I rock and roll?","file_exists":true,"major_mime":"image","media_type":"BITMAP","mime":"image/jpeg","minor_mime":"jpeg","size":608111,"title":{"mArticleID":46687,"mBacklinkCache":{},"mCascadeRestriction":null,"mCascadeSources":[],"mCascadingRestrictions":[],"mDbkeyform":"Rock-and-roll.jpg","mDefaultNamespace":0,"mFragment":"","mHasCascadingRestrictions":null,"mInterwiki":"","mLatestID":false,"mLength":-1,"mNamespace":6,"mNotificationTimestamp":[],"mOldRestrictions":false,"mPrefixedText":"File:Rock-and-roll.jpg","mRedirect":null,"mRestrictions":[],"mRestrictionsExpiry":{"create":"infinity"},"mRestrictionsLoaded":false,"mTextform":"Rock-and-roll.jpg","mTitleProtection":false,"mUrlform":"Rock-and-roll.jpg","mUserCaseDBKey":"Rock-and-roll.jpg","mWatched":null},"url":"/w/uploads/2/2b/Rock-and-roll.jpg","user_id":8306,"user_text":"Ralph"},"msg_id":"2014-95c29490-5e21-48c6-a4c4-c0c5f7aa0bd4","timestamp":1416001297,"topic":"org.fedoraproject.stg.wiki.upload.complete","username":"apache"} }}}
Login to comment on this ticket.