User:NeilK/Worklog/2010-12-14 to 2010-12-20

Projects: Upload Wizard, MediaStorage, Resource Loader

Status: solid

Last week:

• Mostly OOO sick
• Worked on MediaStorage
• Working on UploadWizard patches for next week (Tuesday)
  • didn't have a patch ready, instead worked on the concurrency bugs since they indicate a deeper issue

This week:

• UploadWizard bugs
• Able to help with Resource Loader a bit

Tuesday Dec 14 2010

• Conference call re: MediaStorage
• Weekly Features meeting
• Committed some refactors to HTMLForm, along with a way to list and clear all stashed files from UploadStash. This was a feature I was trying to add Friday in an oh-wouldn't-this-be-helpful-while-debugging mood and it took way too long. Upside, at least now I understand MediaWiki's HTMLForm and other HTML classes, since I was trying to do everything the right way. Now, back to Roan's concurrency patch, which was the whole point of this.

Wednesday Dec 15 2010

• Figured out the concurrency issue and decided that we cannot actually fix this problem at the moment. I fiddled with Roan's idea of using separate memcache items to store everything and it works, although that switches us over to a completely new (and even more fragile) way of storing temporary upload data. That wouldn't be so bad, but we also rely on existing public API methods for the final steps of publishing, which all assume we are stashing files in the old way. So: we lose, unless we also rewrite those API methods, which aren't *quite* expired...

The right thing is to rearchitect all of MediaWiki to use a database table for stashed upload data. For many reasons -- reliability, concurrency, cross-colo functionality. Sessions and memcache are for stashing stuff that's slightly expensive to calculate, not for building apps on top of. Also, we can't impose any policy on sessions or memcache, like "must retain this data for as long as it takes to upload the file" -- for all we know the upload could take hours.

Deferring that until later. In the meantime we can do one upload at a time. (We could also rejigger the front end so that no upload was ever started within one or two seconds of each other, but that mitigates the problem, doesn't solve it -- you can't control timing with HTTP APIs).

Thursday Dec 16 2010

Worked on the highest priority frontend bugs. Fixed Safari 3.2 (and probably a lot of other legacy browsers in the process; there was a javascript backwards incompatibility). Oddly I don't have any environment where I can run Mac Safari 3.2 (Snow Leopard), but I realized it's also available for Windows. I hope the issue was the same.

It would have been nice if Calcey had noted which Safari was actually failing. I couldn't replicate the issue in 5, for obvious reasons. Then I remembered to check their test plan; I forgot they were going to test in a slightly outmoded version of Safari.

Set up a good environment to test IE8, and also got a good handle on why IE8 fails. No fix for the latter yet.

Friday Dec 17 2010

Bryan Tong Minh noticed my uploadstash-backend branch and said that if I'm rewriting it, it should be kicked out of 1.17. Makes sense, but perhaps I could accelerate writing the db version instead.

Dropped into the MogileFS IRC channel to ask a couple of questions, turned into a bit longer session with main developer (Dormando). Added a lot of info to MediaStorage Distributed Filesystem Choices.

Monday Dec 20 2010

Moved desks.