<!DOCTYPE html><html lang="en"><head><meta http-equiv="Content-Type" content="text/html charset=UTF-8"><meta charset="UTF-8"><meta name="viewport" content="width=device-width"><meta name="x-apple-disable-message-reformatting"><title>TLDR Data</title><meta name="color-scheme" content="light dark"><meta name="supported-color-schemes" content="light dark"><style type="text/css">
:root {
color-scheme: light dark; supported-color-schemes: light dark;
}
*,
*:after,
*:before {
-webkit-box-sizing: border-box; -moz-box-sizing: border-box; box-sizing: border-box;
}
* {
-ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%;
}
html,
body,
.document {
width: 100% !important; height: 100% !important; margin: 0; padding: 0;
}
body {
-webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; text-rendering: optimizeLegibility;
}
div[style*="margin: 16px 0"] {
margin: 0 !important;
}
table,
td {
mso-table-lspace: 0pt; mso-table-rspace: 0pt;
}
table {
border-spacing: 0; border-collapse: collapse; table-layout: fixed; margin: 0 auto;
}
img {
-ms-interpolation-mode: bicubic; max-width: 100%; border: 0;
}
*[x-apple-data-detectors] {
color: inherit !important; text-decoration: none !important;
}
.x-gmail-data-detectors,
.x-gmail-data-detectors *,
.aBn {
border-bottom: 0 !important; cursor: default !important;
}
.btn {
-webkit-transition: all 200ms ease; transition: all 200ms ease;
}
.btn:hover {
background-color: #f67575; border-color: #f67575;
}
* {
font-family: Arial, Helvetica, sans-serif; font-size: 18px;
}
@media screen and (max-width: 600px) {
.container {
width: 100%; margin: auto;
}
.stack {
display: block!important; width: 100%!important; max-width: 100%!important;
}
.btn {
display: block; width: 100%; text-align: center;
}
}
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
a {
color: inherit !important; text-decoration: underline !important;
}
</style><!--[if mso | ie]>
<style type="text/css">
a {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
a {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
</style>
<![endif]--></head><body class="">
<div style="display: none; max-height: 0px; overflow: hidden;">Five months after rebuilding its warehouse product on DuckDB, PostHog is moving off it. Each instance had to be sized before the query arrived </div>
<div style="display: none; max-height: 0px; overflow: hidden;">
<br>
</div>
<table align="center" class="document"><tbody><tr><td valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" class="container" width="600"><tbody><tr class="inner-body"><td>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr class="header"><td bgcolor="" class="container">
<table width="100%"><tbody><tr><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" style="margin-top: 0px;" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div style="text-align: center;">
<span style="margin-right: 0px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/LFMMr9eiqjgfnxDecJfY-ooellfI0sM8Uhz0ReV_65M=452" rel="noopener noreferrer" target="_blank"><span>Sign Up</span></a>
|<span style="margin-right: 2px; margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisetopnav/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/5Vy4EXf8sIjtrHIm9kQ6m52yXag6lGfwmU3xMbTbi7Y=452" rel="noopener noreferrer" target="_blank"><span>Advertise</span></a></span>|<span style="margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Fweb-version%3Fep=1%26lc=aa6c57ca-be20-11f0-8e04-b5caa3a65826%26p=1a333d88-d8d0-11f1-8a82-df59037e6684%26pt=campaign%26t=1793877789%26s=b99ed19ce130beb95dce44e463d6e93bcc3554b62041c4eb9ecd563198794374/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/4zAwwfCbLngxPSYMwPu8Lt4QXQfYDVp3HIfGfhGf1W4=452"><span>View Online</span></a></span>
<br>
</span></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="text-align: center;"><span data-darkreader-inline-color="" style="--darkreader-inline-color:#3db3ff; color: rgb(51, 175, 255) !important; font-size: 30px;">T</span><span style="font-size: 30px;"><span data-darkreader-inline-color="" style="color: rgb(232, 192, 96) !important; --darkreader-inline-color:#e8c163; font-size:30px;">L</span><span data-darkreader-inline-color="" style="color: rgb(101, 195, 173) !important; --darkreader-inline-color:#6ec7b2; font-size:30px;">D</span></span><span data-darkreader-inline-color="" style="--darkreader-inline-color:#dd6e6e; color: rgb(220, 107, 107) !important; font-size: 30px;">R</span>
<br>
</td></tr></tbody></table>
<br>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr id="together-with"><td align="center" height="20" style="vertical-align:middle !important;" valign="middle" width="100%"><strong style="vertical-align:middle !important; height: 100%;">Together With </strong>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-pp445qepfdy34%3Ftrk=04612966-5352-4c63-92ba-50274910423a%26sc_channel=el%26source=mongodb%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-11-05_primary_aws%26utm_content=header_ai_amnesia_build%26utm_term=2026-11-05/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/OA_K9C3idb4RnkH80PaW1nAXK6WnUUXU9Q1yQhCcoHo=452"><img src="https://images.tldr.tech/mongodb.png" valign="middle" style="vertical-align: middle !important; height: 100%;" alt="MongoDB"></a></td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;">
<div style="text-align: center;">
<h1><strong>TLDR Data <span id="date">2026-11-05</span></strong></h1>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr id="sponsy-copy"><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-pp445qepfdy34%3Ftrk=04612966-5352-4c63-92ba-50274910423a%26sc_channel=el%26source=mongodb%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-11-05_primary_aws%26utm_content=header_ai_amnesia_build%26utm_term=2026-11-05/2/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/p9_lisVMAC_CYf7P5tYs7VmaHYI6jt8k3_WRNrmMK9Q=452">
<span>
<strong>Does your AI have amnesia? Build agents that never forget with MongoDB Atlas on AWS (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
When your session state, long-term history, and embeddings are scattered in different systems, your AI app needs to start every session from scratch.<p></p><p><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-pp445qepfdy34%3Ftrk=04612966-5352-4c63-92ba-50274910423a%26sc_channel=el%26source=mongodb%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-11-05_primary_aws%26utm_content=Body_mongodb_atlas_aws%26utm_term=2026-11-05/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/eOaMc9KusTUXdcZCKlU2pP0baLjVXiGH1Pyxqle87xc=452" rel="noopener noreferrer nofollow" target="_blank"><span>MongoDB Atlas on AWS</span></a> unifies operational data, vector search, and memory on one platform - so your apps can remember, reason, and act on the same trusted context, every single time.</p>
<p>→ Native vector search and ACID transactions in one database
<br>→ Integrates with Amazon Bedrock and 70+ AWS services
<br>→ 99.995% uptime SLA with multi-region auto-scaling</p>
<p>And if you're on AWS, you can <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-pp445qepfdy34%3Ftrk=04612966-5352-4c63-92ba-50274910423a%26sc_channel=el%26source=mongodb%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-11-05_primary_aws%26utm_content=Body_atlas_aws_marketplace%26utm_term=2026-11-05/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/QdmogWMBq9XEmtxLIwOe_J8cWXc2Ji7ie6ClLnrpM3s=452" rel="noopener noreferrer nofollow" target="_blank"><span>get Atlas on the AWS Marketplace </span></a>with PAYGO or custom pricing and dead-simple onboarding.</p>
<p><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-pp445qepfdy34%3Ftrk=04612966-5352-4c63-92ba-50274910423a%26sc_channel=el%26source=mongodb%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-11-05_primary_mongodb%26utm_content=cta_aws_marketplace%26utm_term=2026-11-05/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/NzQIDvpoake3NTszkKvWnPlgpDcK-R7Doojtm-vWN0s=452" rel="noopener noreferrer nofollow" target="_blank"><span>Start free on AWS Marketplace →</span></a>
</p>
</span></span></div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr>
<tr bgcolor=""><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">📱</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Deep Dives</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fposthog.com%2Fblog%2Fmoving-off-duckdb%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/XO3VJgo8YVM5m3RKYun60mfjQL2XAk-gLyESD5pagdg=452">
<span>
<strong>Why we're moving our warehouse product off DuckDB (5 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Five months after rebuilding its warehouse product on DuckDB, PostHog is moving off it. Each instance had to be sized before the query arrived, and its DuckLake, holding about 1 trillion rows, could take over three minutes for a single-table write. PostHog is now rebuilding on Trino with its own table format, HogLake.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fairbnb.tech%2Fai-ml%2Ffrontier-quality-at-1-of-the-cost-a-production-benchmark-of-llm-classifiers%2F%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/FuA5Tc20LwSiumBM_Di-lPmQkvYlmIUu2jHECpymzCU=452">
<span>
<strong>Frontier quality at 1% of the cost: a production benchmark of LLM classifiers (8 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Airbnb fine-tuned a model with 2 billion effective parameters to screen content for pricing-policy violations. Its F1 score rose from 0.563 to 0.902, matching the best frontier APIs, with a median latency of 35ms against their roughly 900ms. At thousands of requests a minute, the frontier models cost two orders of magnitude more than one self-hosted GPU.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fengineering.fb.com%2F2026%2F11%2F03%2Fml-applications%2Ff3-lightning-how-meta-accelerates-ads-ml-features-to-production%2F%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/adr4XwHXn-GszmE_4xZPSUgeQ1qlF0yVCwi_rTcNVYs=452">
<span>
<strong>F3 Lightning: How Meta Accelerates Ads ML Features to Production at Scale (8 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Meta rebuilt its ads ML feature infrastructure to deploy features independently without losing the efficiency of shared computation. F3 Lightning uses stable computation IDs, a unified execution plan, and shared memory to avoid redundant processing across more than 20,000 features. Comparable features now reach production 3.6x faster without increasing infrastructure compute costs.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fblogs.halodoc.io%2Fupgrading-redshift-from-ra3-to-rg-and-achieving-faster-queries-at-32-lower-cost%2F%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/dyOGIxR_jksyjyMW2YdcPTKkIaBEN9_lzAv-xFSWVl8=452">
<span>
<strong>Upgrading Redshift from RA3 to RG and Achieving Faster Queries at 32% Lower Cost (8 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Halodoc migrated its Redshift warehouse from RA3 to Graviton-based RG instances after validating ingestion, ETL, dashboards, and governance compatibility. COPY operations ran roughly 6x faster, heavy reporting queries ran 2.1x to 5.2x faster, and queue times fell by around 70%. Overall Redshift costs dropped 32%, demonstrating the potential benefits of the new architecture for production analytics workloads.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">🚀</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Opinions & Advice</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.ssp.sh%2Fblog%2Fhumans-write-less-code%2F%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/MhgxLDkdAhFHa_kW9vZAkWWdYBso-P8JKv2EEGbHaPo=452">
<span>
<strong>As Humans Write Less Code: Workloads Shift from Human to Agent Verification with Iterative Test Suites (14 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
As AI agents generate more SQL, transformations, and dashboards, verifying their output becomes more important than writing the code itself. What dev teams really need are compiler-style validation, layered test suites, and unified lineage across data models and BI assets. Moving assertions earlier in development could make agent-generated analytics easier to audit and less prone to semantic drift.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjoereis.substack.com%2Fp%2Fthe-great-data-interregnum-morbid%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/aBO-9rURe1mcKSqw-1S8UnGYyKfdoILlkDflbagcW48=452">
<span>
<strong>The Great Data Interregnum - Morbid Symptoms of an Industry Between Eras (14 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
The data industry is caught between vendor consolidation, AI-driven automation, and unresolved operational problems. Generating SQL, pipelines, and documentation is becoming cheaper, but broken data, inconsistent business definitions, and unclear accountability remain. The lasting value of data teams lies in semantics, governance, and understanding rather than producing more technical artifacts.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.counting-stuff.com%2Fexploratory-data-analysis-cant-be-automated-away%2F%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/3JLuL7Xs7V2EVa2BlOopVDKi96xSvCBUozPOGiwjspM=452">
<span>
<strong>Exploratory Data Analysis Can't Be Automated Away (8 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Exploratory data analysis still needs human eyes on the data. What looks weird depends on what you intend to do, which no model can know in a vacuum. LLMs rarely make the work faster.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">💻</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Launches & Tools</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.datadoghq.com%2Fresources%2Fai-age-data-engineering-market-survey%2F%3Futm_source=tldrnewsletter%26utm_medium=newsletter%26utm_campaign=dg-content-ebook-2026gatherhqdataengineeringtrends-dataobservability-multi-ww-en-701vy00000qux9fyac-tldr-data-110526/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/qgHE4ZEu_L7OZuH_z0xaNIHKsJhNZfRdZEu5i_7Yw78=452">
<span>
<strong>Data engineering in the age of AI: the practitioner view (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
A <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.datadoghq.com%2Fresources%2Fai-age-data-engineering-market-survey%2F%3Futm_source=tldrnewsletter%26utm_medium=newsletter%26utm_campaign=dg-content-ebook-2026gatherhqdataengineeringtrends-dataobservability-multi-ww-en-701vy00000qux9fyac-tldr-data-110526/2/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/uVMhuXzl0zuhSelatEQxYw-RS6lBJQz-iRhcRIXdAcw=452" rel="noopener noreferrer nofollow" target="_blank"><span>new report by Datadog</span></a>, based on the experiences of 109 data engineering practitioners, reveals the speed in which AI is expanding the data engineer's role, and why many teams feel unprepared. Learn what skills data engineers are prioritizing for 2026 and why 70% still see significant gaps. <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.datadoghq.com%2Fresources%2Fai-age-data-engineering-market-survey%2F%3Futm_source=tldrnewsletter%26utm_medium=newsletter%26utm_campaign=dg-content-ebook-2026gatherhqdataengineeringtrends-dataobservability-multi-ww-en-701vy00000qux9fyac-tldr-data-110526/3/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/KOGDZJOc1VF_6iGxsknQfjiFMWY9Fsaz9inwp2UsQYI=452" rel="noopener noreferrer nofollow" target="_blank"><span>Read the report</span></a>
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fsurrealdb.com%2F4%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/O1LAHl_D9XqKi729J4RHC_VU8yh-4YCImTdxUc77gHc=452">
<span>
<strong>SurrealDB 4 (4 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
SurrealDB 4 rebuilds the query engine with 240+ fixes. It reports 42% faster CRUD operations and projects 22x faster indexed queries, while adding PostgreSQL, MySQL, and MongoDB compatibility. The same engine powers Earth's serverless databases on Cloudflare and SurrealDS' distributed clusters, which confirm writes only after storing them across two zones.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fhudi.apache.org%2Fblog%2F2026%2F11%2F03%2Fapache-hudi-rs-0-6-0-release-announcement%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/sErjaiuOxCUbpu8DhAF5pkz-oiMOpWQUgPCIDBOE07c=452">
<span>
<strong>Apache Hudi-rs 0.6.0: Native Writes from Rust and Python (16 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Apache Hudi-rs 0.6.0 introduces native write support for Rust and Python, allowing applications to create, append, update, upsert, and delete records without Spark or a JVM. Its record-level index enables sub-second small-batch updates on tables containing millions of rows. The release also improves Python read performance by up to 6x while maintaining compatibility with existing Hudi query engines.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fgithub.com%2FHelixDB%2Fhelix-foundry%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/sIX0oQWhY_cVt5iuLRfFOu7sJgj0luOTGikckmk8JKU=452">
<span>
<strong>Helix Foundry: Turning Enterprise Data into an AI-Generated Knowledge Graph (GitHub Repo)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Helix Foundry connects databases, SaaS applications, and files, then uses an AI agent to propose an ontology and load the resulting entities and relationships into HelixDB. The open-source project supports local deployment and is adding consistent PostgreSQL snapshots and graph computation capabilities.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">🎁</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><strong><h1>Miscellaneous</h1></strong></div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fgithub.com%2Frocketride-org%2Frocketride-server%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/XyZY9ZZ6X9pbSk2CPsPoQNnTqkT0qthq00TcDZPfmfQ=452">
<span>
<strong>RocketRide: A C++ Pipeline Engine for AI Workflows (GitHub Repo)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
RocketRide is an open-source AI pipeline engine built around a multithreaded C++ runtime with more than 50 Python-extensible processing nodes. It supports multiple LLM providers, vector databases, and agent frameworks, with debugging and deployment tools integrated into VS Code.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Flinks.tldrnewsletter.com%2FwiZrDP/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/gw5NGRhV-r8Khvdm3icLlzkd6AUbFwjnCb0UZHtkuoc=452">
<span>
<strong>Scaling Model Evaluation at Pinterest with Ray (Part 4) (10 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Pinterest used to run inference with Ray, write predictions to S3, then start a Spark job to read them back and compute metrics. Its new evaluation platform computes partial metrics batch by batch inside the Ray inference job and combines them at the end. That cut end-to-end evaluation time by up to 32% for feature-importance runs.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">⚡</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Quick Links</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=quicklinks11052026%26sp=1D_IncT9MYyJivDaQ9cZGmRx/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/RwLPCr2VB7_9_GIvAUXWzRBcz-4Z37ltQb6Z7nnkAzk=452">
<span>
<strong>Reach 8 million tech professionals reading TLDR (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Ads get ignored on social media but not in TLDR! Reach data engineers, analysts and data leaders where they actually pay attention. <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=quicklinks11052026%26sp=1D_IncT9MYyJivDaQ9cZGmRx/2/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/eJYQ7kjx2_6EAyg_uKoKpDq-mOncSWNsmbmnisYWfYw=452" rel="noopener noreferrer nofollow" target="_blank"><span>Learn more about sponsorship opportunities</span></a>.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Festuary.dev%2Fblog%2Fmulti-tenant-cdc%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/9vowpo77MCySyB___0StgM958CnAo3uVZ87qV9vO0gY=452">
<span>
<strong>Multi-Tenant CDC: Database-per-Tenant SaaS Architecture Guide (20 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
A practical guide to replicating tenant-isolated PostgreSQL databases into shared analytics tables while preserving primary keys, schema compatibility, and failure isolation.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Farxiv.org%2Fabs%2F2611.04318%3Futm_source=tldrdata/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/JTtUP7FYBNQGXs7khr76e0X789cTSS3enI02oihxOxc=452">
<span>
<strong>Text-to-SQL Models Make Choices Without Saying So (14 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
SpecBridge shows that text-to-SQL models often silently choose one interpretation of ambiguous questions, while answering each interpretation separately improves coverage by 29-63 percentage points without reducing precision on clear requests.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td align="left" style="word-break: break-word; vertical-align: top; padding: 5px 10px;">
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to advertise in TLDR? 📰
</p>
<div class="text-block" style="margin-top: 10px;">
If your company is interested in reaching an audience of data engineering professionals and decision makers, you may want to <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisecta/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/xgO7riACuHh3jz0ZYdYVoZ3yd3nG6gsSQJ7Vw0HTdbs=452"><strong><span>advertise with us</span></strong></a>.
</div>
<br>
<!-- New "Want to work at TLDR?" section -->
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to work at TLDR? 💼
</p>
<div class="text-block" style="margin-top: 10px;">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjobs.ashbyhq.com%2Ftldr.tech/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/ViwH0YXdx5Yqv9mypQ5GY7O1xfRE3M1FMhAVX1XIqx0=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>Apply here</strong></a>,
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjobs.ashbyhq.com%2Ftldr.tech%2Fc227b917-a6a4-40ce-8950-d3e165357871/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/SnnMaIGfLdERSaWHFx53brquZVMInCy2ZpF0ehGkM0U=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>create your own role</strong></a> or send a friend's resume to <a href="mailto:jobs@tldr.tech" style="color: #0000EE; text-decoration: underline;">jobs@tldr.tech</a> and get $1k if we hire them! TLDR is one of <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Ffeed%2Fupdate%2Furn:li:activity:7401699691039830016%2F/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/IqBT8-piwv6c5q7gA1g1EtAR6cy2q1BCh4-erw9i2Z8=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>Inc.'s Best Bootstrapped businesses</strong></a> of 2025.
</div>
<br>
<div class="text-block">
If you have any comments or feedback, just respond to this email!
<br>
<br> Thanks for reading,
<br>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjoelvanveluwen%2F/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/KxungpTwZWltkODK-Or-BK4swdsQfKnvRd-5uLzOhpg=452"><span>Joel Van Veluwen</span></a>, <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjennytzurueyching%2F/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/2e1ZeZKWVLaBAVOJLUEBuBJqdIOmHv_O10Xrp6Tn9g4=452"><span>Tzu-Ruey Ching</span></a> & <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fremi-turpaud%2F/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/Qnm-umjIlJFljnDp9tAxSSQw_dEtYTP3Wou5us-cS58=452"><span>Remi Turpaud</span></a>
<br>
<br>
</div>
<br>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block" id="testing-id">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%2Fmanage%3Femail=donut.sweater.23%2540rss.andrealin.it/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/lLk53n67xaRfidwPzYq4Uvgnfh16fuL-2JmqFWfzYDk=452">Manage your subscriptions</a> to our other newsletters on tech, startups, and programming. Or if TLDR Data isn't for you, please <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Funsubscribe%3Fep=1%26l=037ede50-92cc-11ee-b0f2-b761aa2217ad%26lc=aa6c57ca-be20-11f0-8e04-b5caa3a65826%26p=1a333d88-d8d0-11f1-8a82-df59037e6684%26pt=campaign%26pv=5%26spa=1793876439%26t=1793877789%26s=8c52b0745258831c62696dfd5e228e5b25643155f6fef5ef5c7caec41ed8544b/1/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/PxO53lbmBrmiDl8l65Tk7l92w3sJ4o76UQtMq-tRNPo=452">unsubscribe</a>.
<br>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
<img alt="" src="http://tracking.tldrnewsletter.com/CI0/010001a1ab72cb28-8bd9c9cb-1e6e-4be4-b947-8af5bc548ef7-000000/qZwfZUgDC-Nm81SBt6LmQPWhqd3X9RrUalQKoaxIjlg=452" style="display: none; width: 1px; height: 1px;">
</body></html>