Software design is not trivial. For this particular project, I decided to start with designing the API and data structures to go with this project. I'm not sure how exactly to document my UI design, so hopefully I can get back to that later.

I've added a mindmap for the required data, and started writing up an Apiary.io API Blueprint, and added them to github. Very little is final yet, but it feels like I might actually get somewhere before the end of next week.

There are three things the API needs to be able to handle immediately: (1) Moderate/High-volume data logs (thousands of writes per second), (2) A concept of a "session" to group logs together, and provide metadata around the logs, and (3) A way to organize those sessions.

Logging is mostly a matter of structuring your data correctly, and using an existing Database Server. Riak, CouchDB, PostgreSQL, and HBase all provide something to the discussion, and for the immediate future, which one of those I choose will not matter very much, as long as I shard the logs.

Sessions are also "just another object to juggle."

To organize sessions, I've provided a way for users to define projects. This allows the user to tell the system which project they're working on, and then either browse sessions or create a new one. Once I get to searching logs, this will also allow some extra context around the sessions that match the search.