51eshare's Tech Blog
This site is dedicated to sharing IT-related technologies.
  Post Categories
Architecture 35 Observability 23 Backend Development 17 Backend Architecture 15 Distributed Systems 33 Cloud Native 22 Software Architecture 15 Security 3 MLOps 17 Cloud Native & DevOps 1 DevOps 35 Data Engineering 41 Web Development 2 API Gateway 1 Mobile Development 3 Software Engineering & Architecture 1 Security Architecture 1 System Design 9 AI Engineering 2 Frontend Architecture 1 Platform Engineering 3 Build Tools 1 System Architecture 6 Real-Time Systems 1 Resilience Engineering 3 DevSecOps 1 MLOps Engineering 1 Cloud Native Security 1 Backend Engineering 2 Cloud Architecture 4 Full-Stack Development 2 Rust Engineering 1 Build Systems 1 Frontend Engineering 2 WebAssembly 1 Security Engineering 2 Cloud Security 1
Implementing a Tiered Data Storage Architecture for an Embeddable RAG System Implementing a Tiered Data Storage Architecture for an Embeddable RAG System
The initial attempt to build our embeddable RAG (Retrieval-Augmented Generation) engine within a Java desktop applicatio
2023-11-15 Data Engineering
LevelDB ChromaDB RAG Java SQLite JPA
Declarative Management of a Stateful Flink Security Pipeline on AWS EKS Using Chef Declarative Management of a Stateful Flink Security Pipeline on AWS EKS Using Chef
The transition to passwordless authentication using WebAuthn eliminated entire classes of credential stuffing and phishi
2023-10-27 Data Engineering
Chef Apache Flink Security WebAuthn AWS EKS Real-time
Implementing a Kafka-Triggered MapReduce Framework for Batch Data Aggregation into MariaDB Implementing a Kafka-Triggered MapReduce Framework for Batch Data Aggregation into MariaDB
Our primary reporting database, a heavily indexed MariaDB instance, was buckling under the load of nightly aggregation j
2023-10-27 Data Engineering
MariaDB Kafka ETL Java MapReduce Scrum
Implementing a Declarative Data Synchronization Fabric Between HBase and DynamoDB via Crossplane Implementing a Declarative Data Synchronization Fabric Between HBase and DynamoDB via Crossplane
The technical directive was deceptively simple: establish a presence for our core data platform in a new AWS region. The
2023-10-27 Data Engineering
Kubernetes DynamoDB HBase Crossplane C# Infrastructure as Code
Architecting a Data Lakehouse Meta-Observability Plane with Apache Iceberg, Prometheus, and a Svelte Headless UI Architecting a Data Lakehouse Meta-Observability Plane with Apache Iceberg, Prometheus, and a Svelte Headless UI
The operational visibility of our large-scale data lakehouse, processing petabytes daily, has become a critical blind sp
2023-10-27 Data Engineering
Headless UI Observability Delta Lake Prometheus TSDB Apache Iceberg Svelte Data Architecture
Implementing a Fault-Tolerant Stateful Aggregation Layer in Dask with LevelDB Backends Implementing a Fault-Tolerant Stateful Aggregation Layer in Dask with LevelDB Backends
The core challenge with large-scale, long-running stream aggregations isn’t the computation itself, but the management o
2023-10-27 Data Engineering
Distributed Systems Dask LevelDB BASE Python
1 / 7