Infrastructure Migration and Automation Initiative
- Mark Breedlove (Unlicensed)
Purpose
To move our servers and other IT infrastructure from Rackspace to Amazon Web Services. To improve the efficiency with which we can manage these resources. To make it quicker and easier to deploy updates to our services.
Audience
Everyone. This affects the availability of the sites, for end users. It affects anyone who benefits from less site downtime and faster responses from the site. It affects us financially, and, in turn, that affects our donors in that it gives them more for their money. It also benefits us, per the next section.
Importance to the DPLA
This is important to us because it improves our ability to schedule staff time to work on other site features, because of the fact that more money can be spent on staff hours for other projects, and less time is required to do various maintenance tasks.
Importance to Digital Library and Other Communities
The software product and expertise that we generate from this project are shared online. Other institutions or interested individuals can implement our software or methodologies. People who want to become involved with our software can download our code from GitHub and spin up their own virtual machines on their own computers that emulate our servers to see how they work. Developers can work with our code in an environment that's already set up correctly, without a lot of effort to get started. The cost-saving benefits mean that we can provide higher levels of API and website responsiveness to our user community per dollar received to run the old architecture. Likewise, our partners will benefit from faster ingestions and faster onboarding.
Differences to Previous Technologies
The legacy architecture was about twice the cost of the current one, per month. The legacy architecture was not consistently automated, and there were no automated deployments of our services.
Timeline for Completion
The project is mostly complete, but we have mailing list and issue tracking services that we hope to migrate to AWS by Summer of 2015.
Opportunities for Community Involvement
Anyone is welcome to check out our automation project on GitHub. (Official Releases | latest development code) Being hosted in a public repository, it is open to pull requests and bug reports from the public.
Following the README instructions in that project, an interested community member can set up virtual machines that run our services.
Although they are a great learning resource and an aid to developers who want to contribute to our projects, our configuration-management scripts and files are designed primarily to support our own operations. That said, interested parties should get in touch with us if they would like to discuss ways to make significant contributions to the project.
DPLA Staff Roles
Mark Breedlove has done most of the DevOps / sysadmin work, including contributions to the automation project and setup of the new Amazon servers. He can answer technical questions about these things. Mark Matienzo and Tom Johnson have also contributed to the automation project code.
All of the tech team members use the automated deployment features to push changes to the live site, and should be able to answer questions about particular services and how the automation works with them.
External Partners and Funders, Grants, etc.
No funders have provided direct assistance, nor are we actively working with any partners.
Project Documentation
There have been official communications with our partners, Board of Directors, and Technical Advisory Committee (via an open conference call) about the AWS transition, and we gave a presentation at Code4Lib 2014 (slides | video); but there is, otherwise, no public document covering the overall initiative.
The automation project's documentation is in its README at the moment. We plan to put more information into Confluence.
Contact