Thank you! The landing page seems to be working; it's drummed up a good deal of interest and people are eager to start using the product.
Development is on track. I currently have data for around 763,000 UK businesses starting from 2015/04/01, and I'm on target to be up to date with 1.1 million this weekend.
The tech stack is
- Haskell with Yesod
- Elm
- Redis
- PostgreSQL
I chose Haskell for the backend because it's one of the safest and fastest languages available. I'm investing my time into building a product business so that I have more time to enjoy my life and learn more. I'm not trying to create another full-time job for myself. My experience working for years with dynamic languages has shown that projects of all sizes and levels of complexity come to a point where someone is spending the majority of their time fighting fires.
I'm currently writing the core of the UI in Elm, which compiles to JavaScript. Again, it's safe and fast, which to me is super important. Safe, because I don't trust my customers to be patient if there are runtime errors. I think they'll want stuff to always just work. And fast, because I'm potentially rendering thousands of nodes in the DOM for a given query. Elm is significantly faster than React, for those curious.
This is my first time relying heavily on Redis, but it's working well. Because all of the data is held in memory, queries are lightning fast. I've heard
rumours of data loss in Redis, but this is not a concern for me, since I'm only using it for company data, which I can reconcile at any time in a background job.
Speaking of background jobs, I'm actually
not using Redis for this. Instead I'm justing using Haskell's software transactional memory so I can mutate state atomically.
All customer data (which is precious!) is stored in PostgreSQL.
I've been monitoring the system for the past week with EKG for Haskell's garbage collection runtime statistics, and collectd for system stats. All of this data is sent to Carbon, and then rendered with Grafana. Memory usage has generally been constant. The system is very stable. CPU usage is spiky because I process as much data as I can until I hit the UK government's rate limiter. I then pause for a minute, and start again.
Here's how that all looks: