Behind the Scenes: The Coolspotters Stack
By Jon Ferry on 19 August 2009
I always find it interesting to see how start-ups handle growth and scaling. That's part of the reason why I joined the Fanzter team, so I could see first-hand the analysis, planning, and execution behind scaling technology. I thought I'd take some time to give back to the community and share the current technology stack behind Coolspotters. Moving forward, we'll be using this space to discuss the pros/cons of new technologies we're researching and the decisions we make along the way.
Where We Are
Coolspotters, our flagship product, was built using Ruby on Rails and runs on Phusion Passenger application servers. Traffic is balanced using HAProxy. We use MySQL for storing all spot, profile, comment, and user data; everything except images. To increase performance and reduce load on our databases, we aggressively cache our data using Memcached. About 50% of Coolspotters activity hits cache. All searches, including site search and form auto-completes, are run against a Solr search server.
All Coolspotters servers are hosted on Amazon's EC2 service. This allows us to scale quickly and perform reliably while keeping our costs down. All MySQL data is stored using Amazon's EBS which provides snapshots several times a day. So should anything bad happen, we can restore our database quickly. I mentioned above that all data except images was stored in MySQL. For images, we use S3 for storage and Amazon's CloudFront for delivery from several locations around the globe. All of our Amazon services are configured, deployed, and managed using RightScale.
Future Plans
We're always trying to create a better experience for our users. This involves careful analysis of our infrastructure, identifying areas that need improvement, creating solutions, and then executing. One area we've identified as needing improvement is search. The indexing time of our current solution, Solr, isn't optimal. Since search is tied into so many aspects of Coolspotters, we know we need to improve this to get the user experience we want. We've been experimenting with Sphinx as a replacement and have been pleased with the results so far.
Another area we are looking to optimize is image encoding, storage and delivery. Currently, most of the media uploaded to Coolspotters are images, so any improvement we make to speed up encoding and rendering time has a great impact on the entire site.
We're looking forward to sharing more about the technical aspects of our products and what goes on behind the scenes here at Fanzter.
-Jon