Amazon MSK customers

Kafka has been a central component of our Engineering Analytics Platform at Arm for several years now, both on prem and in the cloud. Since its initial commissioning we have run it on Arm architecture, however to improve reliability and reduce management overhead we were looking to use Amazon Managed Streaming for Apache Kafka (MSK).

With the enhancement of MSK to support Graviton 3, we are able to return our analytics service to a fully Arm-based stack, delivering higher performance at lower cost while reducing the carbon footprint of our operations.

Amazon MSK allowed us to scale seamlessly while benefiting from enterprise-grade reliability and security. By offloading Kafka management to AWS, we could stay focused on our core business and innovate faster.

It would take 10 hours to synchronize our metadata stores. But now, using Amazon MSK, this process takes 30 minutes.

When migrating away from our datacenter and into AWS, we needed a way to retain our business critical investments in Kafka while significantly reducing the overhead of running Kafka ourselves. This led us to MSK. We’ve shifted a significant portion of infrastructure and Kafka software management onto MSK clusters, which have scaled to support over 40 GB/sec on average of data ingestion. For example, we no longer have to manage hardware and software for keeping our clusters up to date. As a result, we are able to invest more of our resources towards developing services around Kafka for our client teams, which is what we do best. Consuming MSK lets us focus on our core competencies; enabling us to release better products faster and more frequently.

Since adopting Amazon MSK, we’ve been able to significantly increase the efficiency of our teams and reduce time spent maintaining our clusters. As our data requirements and use cases grew, we started to experience considerable overhead and complexity in managing our Kafka clusters. Using Amazon MSK has allowed us to manage our clusters with confidence, spend less time verifying the performance of the clusters, scale with ease, and focus more time on innovating and building exciting real-time applications.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) has made it easy to setup, maintain, and scale Kafka clusters, enabling an end-to-end ingestion pipeline supported by a fully-managed service. As our demands increase, we feel confident that we will be able to scale at ease and expand our streaming ingestion pipelines for data enrichment and real-time machine learning, providing an enhanced experience for our customers.

By using Amazon MSK, Vonage has reduced our maintenance overhead while building a micro-services-based platform that enables businesses to collaborate more productively and engage their customers more effectively. We can now set up new Kafka clusters more quickly and scale these clusters as our demands increase. Amazon MSK has removed the complexity of set up and maintenance, allowing us to focus on what’s most important - building innovative new capabilities for our customers.

When developing and launching our security analytics application, Secureworks chose Amazon MSK service to accelerate time to market and reduce infrastructure and management overhead. This allowed us to focus on innovation and our unique differentiators, helping protect our customers from cyber threats.

Streaming technology is an important part of what we do as a business, but managing streaming technology involves a lot of heavy lifting by our software developers. Amazon MSK has allowed us to eliminate a lot of this heavy lifting. By using Amazon MSK we no longer need to spend effort on ensuring data durability, cluster availability, and scalability, and can now focus on building applications. Amazon MSK relieves our team from the burden of operating and maintaining Kafka and at the same time, we experience less downtime as our clusters are fully-managed.

At ZipRecruiter, we use Apache Kafka for our event logging pipelines, which are critical infrastructure components that collect, store, and continually process over 6 billion events per day from the ZipRecruiter employment marketplace. We turned to Amazon Managed Streaming for Apache Kafka (Amazon MSK) to make it easier to manage, run, and maintain this critical infrastructure. Amazon MSK significantly reduced our Apache Kafka management and administrative overhead, which enabled our engineers, who were managing Apache Kafka on a regular basis, to shift focus towards product innovations that make it easier for job seekers and hiring managers to find a great match.

Delhivery uses Apache Kafka as a critical component in their pipeline to collect, store, and logically swivel metadata of First, Mid, and Last mile operations. As we scaled our logistical operations, significant effort was spent managing the infrastructure complexities within our Apache Kafka clusters. Amazon MSK not only helped us offload infrastructure overhead, but we also maintained high throughput and performance for our business critical metadata pipelines in a more secure and reliable manner.

Vortexa uses Apache Kafka to apply Machine Learning models at scale to new and historic data streams and make real-time market predictions immediately available to its customers through its analytics platform. Amazon MSK has been the pivotal technology to Vortexa, and coupled with Lenses.io, enabled us to shift our significant efforts from maintaining and stabilizing a complex and fragile Apache Kafka infrastructure to focusing more on the quality of analytics and market insights that directly impact the value we deliver to our customers. We can now confidently and frequently update our applications at scale, run complex Kafka streams topologies with ease, and debug applications instead of infrastructure. Our revenue and market share are directly impacted by the speed at which we deliver accurate data to market, and Amazon MSK is a key enabler for that.

We use various AWS services to stream and process large amounts of data for live sports, so many parts of our infrastructure have very high utilization on game days and zero utilization on other days. Before using Amazon Managed Streaming for Apache Kafka (Amazon MSK) Serverless, the moving parts and operational overhead required to efficiently scale up and down for game days were a major burden, and capacity was often over-provisioned. Amazon MSK Serverless helped us to offload scaling concerns onto Amazon, which gives us the freedom to spend our engineering time elsewhere. We immediately started saving tens of thousands of dollars on our monthly AWS bill, and were able to reallocate our valuable engineering time from capacity planning and optimizing scaling mechanisms to tackling other tasks.