A build-time summary of 32 unique engineering articles collected across seven daily digests.
Topic Trends
- AI & Machine Learning — 12 articles
- Developer Tools — 11 articles
- Infrastructure & Scale — 4 articles
- Engineering Culture — 3 articles
- Databases & Storage — 2 articles
Source Pulse
- GitHub Blog — 9 articles
- AWS News — 6 articles
- Google Developers — 4 articles
- Stripe Engineering — 4 articles
- Dropbox Engineering — 3 articles
Representative Articles
Building with Gemini Embedding 2: Agentic multimodal RAG and beyond
Google Developers · AI & Machine Learning
Google has announced the general availability of Gemini Embedding 2, a unified model that maps text, images, video, audio, and documents into a single semantic space This model allows developers to process interleaved multimodal inputs in a single request, significantly improving performance for tasks like agentic RAG, visual search, and content moderation By supporting over 100 languages and offering features like task-specific prefixes and Matryoshka dimensionality reduction, the model provides a highly efficient and accurate foundation for building complex AI agents.
Give GitHub Copilot CLI real code intelligence with language servers
GitHub Blog · Developer Tools
Install and configure LSP servers for GitHub Copilot CLI, replacing brute-force grep/decompile with real code intelligence The post Give GitHub Copilot CLI real code intelligence with language servers appeared first on The GitHub Blog.
From Silos to Service Topology: Why Netflix Built a Real-Time Service Map
Netflix TechBlog · Infrastructure & Scale
We needed to combine multiple sources.These lessons shaped every decision we made in building Service Topology.What We Needed: A Living MapWe set out to build something specific: a living map of our infrastructure — one that updates in real-time as services deploy, as traffic patterns shift, as new dependencies form and old ones disappear.The requirements were clear:Real-time updates, not stale snapshots: In an environment where services deploy continuously, yesterday’s topology map is archaeology, not observability.Fast queries at scale: When an engineer is troubleshooting at 3am, they can’t wait minutes for a query to return This graduated approach also prevents hot spots by distributing load across multiple points even when specific applications or network intermediaries see 100x more traffic than others.Graph Storage: We persist the topology in Netflix’s graph database, an abstraction layer built on top of our distributed key-value storage infrastructure Each of our three data sources (network flows, IPC metrics, tracing) creates a separate graph that can be queried independently or merged.gRPC API: We expose the topology through a gRPC service that supports multi-hop traversal, filtering by availability tier and business domain, pagination for large result sets, and sub-second query response times.The technical details of building this at Netflix scale — handling Kafka lag, managing memory and garbage collection, optimizing distributed processing, debugging reactive streams — deserve their own discussion
Dynamic Repartitioning for Time Series Workloads
Netflix TechBlog · Engineering Culture
Using these tools, we can detect cases of both over and under partitioning.Below is an example of over‑partitioning, where the TimeSeries provisioning pipeline selected very small time_bucket intervals based on user provided inputs:Provisioning selected 60s time buckets based on user inputscausing partitions to have less than 10 KB of data, leading to high read amplification and thread queueing:Histogram of the given Cassandra table showing partition size percentilesIn order to tune partition strategies efficiently, we added a background worker, which monitors partition histograms of Time Slices attached to a given application, and exposes it via a Cassandra virtual table:Histograms exposed through a Cassandra Virtual tableIt then computes an adjustment factor when it detects partition sizes not meeting a configured density Simply tolerating elevated latencies or timeouts when querying these IDs is not a desirable outcome.This is where dynamic partitioning comes into play.Solution 2: Dynamic Partitioning per IDDynamic partitioning is an asynchronous pipeline that auto-detects and splits wide partitions on a TimeSeries ID level rather than at the table level.It has three main stages:Detection: Detects wide partitions for a given TimeSeries ID during the read path.Planning & Splitting: Plans and executes splits of those partitions into optimal sizes asynchronously.Serving Reads: Re-routes the read queries transparently to read data from the split partitions when ready.This is how it works at a high level; we will dive into details after:Dynamic Wide Partition Split Async PipelineHere are the different stages of the pipeline:DetectionEvery TimeSeries read operation tracks how many bytes are read for a given partition For example, if EventBucketPartitionSplitStrategy is selected, we split the partition by assigning more event buckets to the same time bucket
High-Throughput Graph Abstraction at Netflix: Part I
Netflix TechBlog · Databases & Storage
Looking ahead, we plan to leverage the graph schema for additional improvements, such as:Minimizing Query Fanout: By using edge cardinality within edge mappings, we aim to select the most efficient traversal paths and minimize query fanout.Improved Developer Experience: The schema will support generating a type-safe data access layer and enhance the Gremlin-like API with schema awareness.Next, let’s look at how this data is organized in a real-time index within the KV Abstraction.Real-Time Index: Key-Value StorageBefore we discuss how the data is organized into graph indexes, let’s discuss how KV organizes data within namespaces and provides idempotency guarantees:Data partitioning: A namespace is associated with a table in the underlying storage layer In order to ensure optimal performance without exerting too much memory pressure, we aim to limit the number of edges per source node within the system.Next, let’s explore the caching strategies used by the Abstraction.Caching Strategies in Graph AbstractionAlthough the Graph Abstraction already provides efficient reads and writes to durable storage, caching remains critical for the stability and performance of any graph datastore for two key reasons:Write amplification: A single write on the fronting service can result in multiple writes to the backing durable storage due to the use of multiple indexes As illustrated in the diagram below, both the caching layer and durable storage replicate data asynchronously across regions, resulting in an eventually consistent system.Now that we’ve covered storing the real-time graph index, let’s see how it enables graph traversals.Graph TraversalsThe Abstraction provides a custom gRPC traversal API, inspired by Gremlin, which enables exploration of the distributed graph by letting users chain traversals, apply filter criteria, sort results, limit results, and more.Let’s explore a hypothetical scenario where the Abstraction is used to recommend shows to users on a shared device, by considering the duration of the most recent viewing session for each show across all profiles and accounts associated with that device:TraversalRequest.newBuilder() .setNamespace("
Method
Articles are de-duplicated by URL, then classified with a deterministic engineering keyword taxonomy. The report is generated during the site build and does not use a database or an external AI API.