Introducing the Quantexa Streaming Best Practice Hub
We're excited to announce the launch of the Quantexa Streaming Best Practice Hub – a curated collection of technical guidance, real-world examples, and expert insights to help you build smarter, faster, and more maintainable streaming solutions on the Quantexa Platform. Whether you're an Engineer, Architect, or Product Owner, this space has been designed to support you throughout your streaming journey. What’s in the Hub? You’ll find a growing set of resources, including: Streaming best practices – Proven architectural patterns, delivery guidance, and maintainability tips from implementations across Financial Services, Government, and other sectors. Deep-dive technical articles – Topics such as Kafka internals, message ordering, stream optimization, backpressure handling, custom scoring apps, and performance tuning. Persona-based guidance – Actionable recommendations tailored to different roles across delivery and platform teams. Solution patterns – Practical examples of how Quantexa streaming supports real-time detection, scoring, enrichment, and resolution use cases. Blog series – Honest, hands-on posts drawn from real implementation experiences and lessons learned in the field. Why this matters? Quantexa streaming is powerful, but great outcomes rely on strong solution design and platform alignment, not just configuration. This hub exists to help you get there faster and more confidently, with guidance built on real-world challenges and delivery experience. It’s also about helping you deliver resilient, scalable, and easy-to-maintain streaming solutions. Where to find it? The Streaming Best Practice Hub is now live on the Quantexa Documentation site and the Quantexa Community. You can jump straight into some key content below: Documentation Site Streaming best practices Planning a streaming solution Designing data ingestion pipelines Deploying a streaming solution Monitoring a streaming solution Debugging and troubleshooting Community Platform Architecture: Kafka Streaming Using Quantexa Kafka Streaming for the First Time Designing a Kafka Solution to Meet Functional and Non-Functional Requirements Data Streaming Design Principles to Enrich Input Messages Lessons Learned from a Streaming Lending Fraud Project Maintaining Message Ordering in Kafka Searching Entities Without Document Ingestion Optimizing Entity Resolution and Graph Expansion Help us grow this? This is just the beginning—we want this hub to evolve based on your needs. If there’s content you’d like to see added or challenges you’d like help addressing, we’d love to hear from you. Your feedback and ideas will directly shape future updates, and contributions are always welcome. To share your thoughts, feel free to leave a comment on this post or email us at community@quantexa.com.Data Streaming Design Principles to Enrich Information in Kafka Input Messages
This article provides practical solution designs to enrich information in incoming messages and align them with Quantexa's Streaming Tier (Document Model) definitions, leveraging lookup principles to ensure seamless integration and efficient operation. The Quantexa Streaming Tier is a critical component for enabling near-real-time data ingestion and processing. Ensuring that input messages adhere to the required schema and include necessary data is essential for maintaining system performance, reliability, and data integrity. Log in to read the full guide on Data Streaming Design Principles to Enrich Information in Kafka Input Messages, including: Conforming Data Using a Mapper Application Data streaming lookup design architecture Lookup considerationsDesigning a Kafka Solution to Meet Functional and Non-Functional Requirements Beyond Quantexa
Designing a Quantexa streaming solution that addresses both functional and non-functional requirements can be challenging, especially when specific use cases require capabilities beyond the OOB functionalities of Quantexa platform. While some extensions may be necessary to meet these requirements, it is important to avoid unnecessary customizations and follow Quantexa’s best practices to ensure the solution remains: Future-proof: Ready for platform upgrades and changes. Aligned with industry standards: Compliant with recognized guidelines and practices. Optimally performing: Delivering high efficiency and responsiveness. Scalable: Easily adaptable to growing data and user demands. Seamlessly integrated: Enhancing operational efficiency within the Quantexa ecosystem. This article explores how to design Kafka streaming solutions that address unique requirements while staying aligned with Quantexa's best practices, minimizing customizations, and leveraging the platform’s capabilities effectively. Read the full article (login required): Designing a Kafka Solution to Meet Functional and Non-Functional Requirements Beyond Quantexa
📚 Kafka Data Ingest
Kafka Data Ingest handles near real-time ingestion of raw Documents into the platform. This enables you to perform ad-hoc Document ingestion and implement larger-scale integrations with event-streamed data sources. Kafka Data Ingest consists of two sets of services, each running within a separate application: Record Extraction service: The Record Extraction service handles the Cleansing, Parsing, and extraction of Records from raw Documents. Document Ingest service: The Document Ingest service loads extracted Records to Elasticsearch for use within Quantexa services and the UI. Read more about Kafka Data Ingest on the Documentation site. Note: Kafka Ingest was referred to as Kafka Loader before version 2.1.