Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Apache Kafka
What Is Apache Kafka? Main Benefits, Use Cases, And Functionalities
Kafka is software that was originally built by LinkedIn (and now managed by the Apache Software Foundation). It allows different applications to reliably send and receive streams of data in real-time. For example, imagine you have a lot of users interacting with your app. Kafka lets you gather all those user events and make them available for other applications to process. Or you could stream IoT data from sensors to Kafka, and from there to data storage and analytics tools.
KIP-932: Queues for Kafka
Operation State changes Cumulative state Starting state of topic-partition with latest offset 100 SPSO=100, SPEO=100 SPSO=100, SPEO=100 In the batched case with successful processing, there’s a state change per batch to move the SPSO forwards Fetch records 100-109 SPEO=110, records 100-109 (acquired, delivery count 1) SPSO=100, SPEO=110, records 100-109 (acquired, delivery count 1) Acknowledge 100-109 SPSO=110 SPSO=110, SPEO=110 With a messier sequence of release and acknowledge, there’s a...
Open Source vs. Open Core
Pure open source technologies are the forward evolution of the software industry. Ongoing acceleration in enterprise open source adoption continues to reaffirm the trendline. The collaborative talent of robust open source communities regularly achieve results that eclipse the strengths – and expose the limitations – of proprietary alternatives, writes Instaclustr CEO Peter Lilley.
APIGEE: CI/CD Pipeline for API Proxies
Home » Java » Enterprise Java » APIGEE: CI/CD Pipeline for API Proxies About Siva Janapati Siva Prasad Rao Janapati is an Architect. He has hands on experience on Java, JEE, Spring, Oracle Commerce, MOZU Commerce, Apache Solr, Apache Kafka, Node.js, JBoss, Hibernate, Memcached, MySql, Oracle, MongoDB, APIGEE, Cloud Native, BlockChain and other open source/enterprise technologies. He loves to explore new technologies and trends.
Introduction to Event Streaming with Kafka and Kafdrop
Making sense of modern architectural trends in data and application engineering Event sourcing, eventual consistency, microservices, CQRS… These are quickly becoming household names in mainstream application development. But do you know what makes them tick? What are the basic building blocks required to assemble complex, business-centric applications from fine-grained services without turning the lot into a big ball of mud? This article examines a fundamental building block — event streaming.
Configuring High Availability and Consistency for Apache Kafka
CDH 6 includes Apache Kafka as part of the core package. The documentation includes improved contents for how to set up, install, and administer your Kafka ecosystem. For more information, see the Cloudera Enterprise 6.0.x Apache Kafka Guide. We look forward to your feedback on both the existing and new documentation.
KIP-499 - Unify connection name flag for command line tool
This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Replication tools
1. Preferred Replica Leader Election Tool What does the tool do? With replication, each partition can have multiple replicas. The list of replicas for a partition is called the "assigned replicas". The first replica in this list is the "preferred replica". When topic/partitions are created, Kafka ensures that the "preferred replica" for the partitions across topics are equally distributed amongst the brokers in a cluster.
KIP-382: MirrorMaker 2.0
Status Current state: Under Discussion Discussion thread: here JIRA: KAFKA-7500 - Getting issue details... STATUS Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast). Motivation MirrorMaker has been used for years in large-scale production environments, but not without several problems: Topics are created with default configuration. Often they'll need to be repartitioned manually.
Optimizing State in a Flink Application With a Map and Reduce Operation
In What is Apache Flink? A Developer’s Guide to Getting Started, the New Relic Account Experience team built our first Flink app, which we call the Usage Calculator. This app receives and processes millions of messages from New Relic APM, Infrastructure, and Synthetics agents every day. The messages come in on Kafka topics and contain data from the agents about how often they’re checking in; specifically, we’re interested in how many hours a day our customers are using New Relic.
Alpakka: Reactive Streams Integrations for AWS, Azure, and Google Cloud [Video]
As the number of systems within an IT infrastructure increases, the number of integrations needed by enterprises also multiplies. Recognizing that the old times of overnight file exchanges are no longer meeting real-time demands, a well-organized enterprise integration strategy is a critical success factor when your systems need to be connected all day.
December 2017 Tracker - Where’s Hadoop?
The leading 2017 story of Hadoop distributions is that nobody seems to want to be accused of being in the business of providing them. Some former champions are expanding their shiny new positioning: Cloudera is selling Enterprise Data Hubs and Analytic DBs; Hortonworks offers DataPlanes and Next-Gen Data Platforms; MapR touts the Converged Data Platform.
System Tools
The source code for Apache Kafka System Tools are located https://github.com/apache/kafka/tree/0.8/core/src/main/scala/kafka/tools If you are looking for the replication related tools then please check out the wiki page on Replication tools System tools can be run from the command line using the run class script (i.e. bin/kafka-run-class.sh package.class --options) Consumer Offset Checker This tool has been removed in Kafka 1.0.0. Use kafka-consumer-groups.sh to get consumer group details.
KIP-98 - Exactly Once Delivery and Transactional Messaging
[This KIP proposal is a joint work between Jason Gustafson, Flavio Junqueira, Apurva Mehta, Sriram Subramanian, and Guozhang Wang] This document outlines a proposal for strengthening the message delivery semantics of Kafka. This builds on significant work which has been done previously, specifically, here and here. at least once semantics, viz. When tuned for reliability, users are guaranteed that every message write will be persisted at least once, without data loss.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Apache
Contact Apache, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack