Job openings/Operations Engineer (Labs)
Operations Engineer [CLOSED]
Director of Operations
As the Operations Engineer for Labs, your primary objectives include the building and operating the new Tools and Labs Virtualization infrastructure using OpenStack and LAMP-stack technology and the on-ramping of the community to use this new platform. You will be integrating with our backend infrastructure such as LDAP, Puppet and MediaWiki, and maintaining the security, scalability and availability of the web properties. In addition, you will be working in tight collaboration with the volunteer community, the rest of the operations team and the other engineering departments.
The Technical Operations team in Wikimedia Foundation is embarking on a new project to build a flexible and scalable tools and lab infrastructure for our community and volunteers, to support their effort to prototype, develop, test and deploy their tools and extensions. Some of the uses of the infrastructure are for:
- Prototyping and staging of WMF-developed MediaWiki code
- Prototyping and staging of volunteer or chapter-developed MediaWiki code
- Deployment of volunteer-created tools which are independent of MediaWiki, e.g. edit counters, mentoring database, geographic information about articles etc. (essentially the kind of things currently running on the Toolserver)
- Development and deployment of new site architecture by staff and volunteers in a code-reviewed, devops oriented environment
- Access for researchers (WMF or external) to live database replication or large datasets, as well as computing resources, for the purpose of running analyses
- Serve as an execution and hosting space for bots, so that bots can be more systematically developed and tracked
Duties include developing, integrating and deploying tools to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. This candidate is also expected to provide technical guidance and support to community-initiated projects. It includes the following:
- On-boarding and shepherding (new) users through the process and infrastructure
- Develop, integrate, test and deploy server infrastructure using OpenStack Compute stack and KVM as the platform
- Integrate with current production infrastructure e.g., LDAP, MediaWiki & MySQL databases
- Integrate & deploy OpenStack Object Storage to provide file storage for images and multimedia content
- Provide system and database administration duties
- Optimize, monitor and maintain the infrastructure and systems, including security, user account and database administrations
- Response to system outages
- Document internal systems and processes
- Use of Puppet to provide configuration management, Git for version code and Gerrit for code review.
Your working language will be English. You must be willing to travel, including a twice yearly trip to San Francisco for all-hands meetings.
- You have advanced knowledge of Virtualization and LAMP-stack technologies and 3+ years of relevant work experience
- You are able to work independently where needed, and can work remotely as part of a globally distributed team
- You are able to learn quickly. Relevant hands-on experience and eagerness to learn and try new concepts is more important than having certificates
- Experience with virtualization technologies such as OpenStack, VMWare or Ganeti
- Experience with configuration management systems and concepts (e.g. puppet, chef, cfengine)
- Experience with operating system distribution packaging systems (e.g. dpkg, RPM)
- Experience with large web site application architectures, including caching layers (memcached, HTTP caching), storage scaling concepts
- Experience with internal infrastructure systems and concepts, e.g. DNS, NTP, LDAP
- Experience with systems programming and development in scripting languages such as Python
- Experience with database administration (MySQL) and general storage systems (GlusterFS or SWIFT)
- Experience with systems and network security is a plus
- Understanding of the free culture movement is a plus
- Experience working with online volunteers is a major plus.
- You are a proficient English speaker
Please include a cover letter to your application. Due to the volume of applications we receive, we regret that only those selected for an interview will be contacted.
Local San Francisco, CA candidates are preferred though remote candidates may be considered.