Date
Attendees
Goals
Discuss R&D projects.
Discuss allocation and make issue tracker tasks for prototype work.
Discuss milestones, deadlines.- Generate list of priorities for what we want our solution to achieve
Discussion items
| Time | Item | Who | Notes |
|---|---|---|---|
Priorities. What are we fixing?
- Speed: speed is a feature. Predictably say how long some ingest will take.
- Allowing recovery from failure; pick up where it left off. Speed affects this; if it's fast enough you don't have to worry about it. Otherwise, make sure there's recovery. Harvesters should allow recovery, where possible. Indexers could also be less speedy than mappings and enrichments, and may deserve recovery features.
- Adding automation that was originally specified: have a program that shepherds the process all the way through. Scheduling.
- Eventually, provide a useable mapping DSL
- Needs real market research
- This is not a turnkey solution yet. Some things like DSLs will be evaluated later when we can more confident in understanding how big the user base is.
- Writing mappings ourselves in the third system without a DSL will allow us to understand the problem space better.
- Ability to debug things, especially mappings
Discussion of code examples
Scheduling and operation chaining
- Need metrics for what qualifies job failure. (Partly thought out)
- Need to get together and assess our experiences running ingests.
- If we automate things, we need to know how to define success.
- Tools exist that can help with this.
- Need to schedule a period after basic manual ingest running is figured out, but need to design for there being a scheduling facility.
Follow these principles:
- https://en.wikipedia.org/wiki/Dependency_inversion_principle
- https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)