“We decided to partner with Onica to really enhance and accelerate our transition. We had developers and engineers from Onica in our office working side-by-side with our teams to help us learn and understand how to leverage the ecosystem in Amazon and help architecturally build the solutions we created. I think the partnership has been one where they are an extension of our team at this point.”
CTO at TrueCar.com
TrueCar innovates faster, scales to meet rapid growth, cuts operational costs by 50% and transforms development
TrueCar is an online digital car buying marketplace that aims to make car buying simple, fair, and fun. The company provides automotive pricing and other data for thousands of buyers of new and used cars in the United States. The organization was experiencing growth challenges with their existing data center technology, so they turned to Onica and Amazon Web Services (AWS).
Price transparency is a critical part of the TrueCar business model, and the company relies on its Vehicle Intelligence System (VIS) to enable that transparency. The system is a central hub that stores raw car pricing data collected from thousands of dealers, transforms and processes the data, and distributes it to dozens of consumer-facing applications.
AWS to the Rescue
As TrueCar grew, managing the VIS became a challenge. “We were running the VIS and other key applications in internal data centers, and the hardware wasn’t keeping pace with the business,” says Tommy McClung, CTO. “We have a small operations team managing thousands of virtual servers. That was getting more difficult to do as we added more dealers to the network.” The company needed to increase the speed of innovation. “We had 100 software engineers sharing the same QA and development environments for testing, and that slowed the pace of development,” says David Giffin, VP of Technology. “Our product velocity had screeched to a halt. As a company, we needed to innovate faster.”
TrueCar Scales Easier with AWS
For TrueCar, moving to the cloud was the only possible solution to its business challenges. “We continued to see more companies successfully migrating key applications to the cloud, and we knew it was the right time for us to do it,” McClung says. After evaluating several cloud providers, TrueCar made the choice to go all in on Amazon Web Services (AWS). “AWS is the category leader, and it had all the services we need to run our application and our company. The choice was very clear for us,” says McClung.
TrueCar worked with Onica, a Premier AWS Partner Network (APN) Consulting Partner who holds multiple AWS Competencies, including the AWS Migration, Big Data, and DevOps Competencies. “We needed outside experts to help us build the cloud platform we wanted to build, and Onica was the right choice,” says McClung. Onica worked with TrueCar to create a sophisticated big data system powered by the Hadoop open-source framework and AWS.
The VIS system’s inventory data—including car make, model, year, and options information—flows through the Hadoop environment, connected to AWS through AWS Direct Connect. The VIS system, several car-buying applications, and partner platforms run on Amazon Elastic Compute Cloud (Amazon EC2) instances, with automobile images stored in Amazon Simple Storage Service (Amazon S3) buckets. Using AWS to power its big data system, TrueCar processes more than 65 billion pieces of data daily, from more than 12,000 data sources, including unstructured data that helps the company detect car-buying trends that show up in clickstreams. The company can also price more than 200 billion separate vehicle configurations every day, with the primary database updated every 30 minutes.
Additionally, TrueCar runs its data pipeline through AWS Lambda and Amazon Kinesis Streams, notifying various car-buying applications when vehicle data is updated and ready for analysis. TrueCar uses Amazon Kinesis Streams to collect and process log data, and Amazon Kinesis Firehose to load the log messages to S3 for permanent storage. TrueCar runs all of its business analytics and reporting using an Amazon Redshift data warehouse.
The company also created internal development and test environments called “space pods.” These containers for self-service development are managed as Docker containers powered by Amazon EC2 Container Service (Amazon ECS), a scalable container-management service. TrueCar software engineers can use the pods to quickly spin up full AWS environments for experimenting with new features. “Each pod contains a full instance of the TrueCar application, and it’s a replica of what’s in production,” says Giffin. TrueCar also uses the Amazon Elasticsearch Service to run search capabilities for its big data system.
By moving more than 1,000 virtual machines from its data center to AWS, TrueCar can now scale to meet demand while no longer needing to manage hardware. “Even though our operations team is small, we can still keep pace with our fast growth by relying on AWS to manage the environment that powers our big data and vehicle-information systems,” says McClung. “And by shutting down our data centers, we anticipate we will lower operational costs by 50 percent in the long run.”
TrueCar can bring new features and products to market faster, because its development and test environments give engineers the flexibility to quickly test ideas. “We are no longer limited by turnaround time for buying and setting up hardware,” says McClung. “It used to take us up to six months to set up a new environment for testing a feature, and if the feature didn’t work, we wasted all that time. Now, using our AWS-powered space pods, we can spin up a development environment in 15 minutes to prototype new ideas. This means we can innovate much faster, which gives us a competitive advantage.”
The company’s flexible new development and test environments have also transformed the way operations staff and application developers work. “Previously, things were very siloed. Our infrastructure team focused on the back end, and our application team focused on software,” says Giffin. “Using AWS, however, we now have an infrastructure-as-code approach that has made us all developers. Everyone can collaborate much easier.”
TrueCar has also significantly improved its data analytics using AWS. “Using Amazon Kinesis Streams and Amazon Redshift, we get near-real-time access to important clickstream data and sales and inventory information,” says McClung. “As we continue to expand our data-analytics platform on AWS, we believe we’ll find new insights from our data to help us drive innovation in the automotive industry.”
Onica is one of the largest and fastest-growing Amazon Web Services (AWS) Premier Consulting Partners in the world, helping companies enable, operate, and innovate in the cloud. From migration strategy to operational excellence and immersive transformation, Onica is a full spectrum AWS integrator. Learn more at www.onica.com.