51eshare's Tech Blog
This site is dedicated to sharing IT-related technologies.
Building a Cost-Aware Node.js ELT Orchestrator for Snowflake Through Iterative Refinement Building a Cost-Aware Node.js ELT Orchestrator for Snowflake Through Iterative Refinement
The initial problem wasn’t a technical marvel; it was a bog-standard operational headache. Our data ingestion pipeline w
2023-10-27 Data Engineering
Node.js Observability Snowflake Scrum TypeScript ELT
Declarative Management of Snowflake Resources Using a Crossplane and GitHub Actions Control Plane Declarative Management of Snowflake Resources Using a Crossplane and GitHub Actions Control Plane
The request came through as it always does: a new microservice needs its own sandboxed database in Snowflake for analyti
2023-10-27 Cloud Native
GitHub Actions GitOps Snowflake RESTful API Crossplane Platform Engineering
Integrating a Go gRPC Streaming Service with a Quarkus Middleware Layer into a Laravel Application Integrating a Go gRPC Streaming Service with a Quarkus Middleware Layer into a Laravel Application
The initial requirement seemed straightforward: add a real-time, AI-powered document analysis feature to our existing La
2023-10-27 Backend Architecture
gRPC-Go Laravel Material-UI (MUI) LLM Quarkus
Implementing a Dynamic Preview Environment System on GCP Using Caddy, esbuild, and Cloud SQL for MySQL Implementing a Dynamic Preview Environment System on GCP Using Caddy, esbuild, and Cloud SQL for MySQL
A shared staging environment is a bottleneck. It’s a single point of failure where feature branches collide, data gets c
2023-10-27 DevOps
GCP CI/CD Caddy esbuild MySQL IDP Cloud Run
Implementing a Git-Driven GraphQL Schema Integrity Validator in C# Implementing a Git-Driven GraphQL Schema Integrity Validator in C#
A production incident was traced back to a seemingly innocuous GraphQL schema change. A field was renamed in one service
2023-10-27 DevOps
GraphQL Client Git CI/CD C# API Governance
Implementing an Archival Pipeline for Event Sourcing Streams with Kafka Connect and HDFS Sink Implementing an Archival Pipeline for Event Sourcing Streams with Kafka Connect and HDFS Sink
The operational event store, a PostgreSQL table holding our core business events as JSONB, was growing at over 500GB per
2023-10-27 Data Engineering
Hadoop Event Sourcing Message Queue Kafka HDFS Architecture
44 / 52