Thursday, September 5, 2024
Wednesday, September 4, 2024
New top story on Hacker News: Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
16 by skull8888888 | 1 comments on Hacker News.
Hey HN, we’re Robert, Din and Temirlan from Laminar ( https://www.lmnr.ai ), an open-source observability and analytics platform for complex LLM apps. It’s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust. How is Laminar different from the swarm of other “LLM observability” platforms? On the observability part, we’re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry ( https://ift.tt/L2stIBP ), open-source package for span production. The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track “semantic metrics” (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span. We think it’s a win to separate core app logic from LLM event processing. Most devs don’t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working. Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it’s overkill for simple pipelines, but it’s extremely cool and we’ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network. One thing missing from all LLM observability platforms right now is an adequate search over traces. We’re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it’s gonna be crucial part of our platform going forward. We also support evaluations. We loved the “run everything locally, send results to a server” approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we’re pushing hard on them. Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it’s current shape, Laminar is just few weeks old and developing rapidly, we’d love any feedback or for you to give Laminar a try in your LLM projects!
16 by skull8888888 | 1 comments on Hacker News.
Hey HN, we’re Robert, Din and Temirlan from Laminar ( https://www.lmnr.ai ), an open-source observability and analytics platform for complex LLM apps. It’s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust. How is Laminar different from the swarm of other “LLM observability” platforms? On the observability part, we’re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry ( https://ift.tt/L2stIBP ), open-source package for span production. The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track “semantic metrics” (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span. We think it’s a win to separate core app logic from LLM event processing. Most devs don’t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working. Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it’s overkill for simple pipelines, but it’s extremely cool and we’ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network. One thing missing from all LLM observability platforms right now is an adequate search over traces. We’re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it’s gonna be crucial part of our platform going forward. We also support evaluations. We loved the “run everything locally, send results to a server” approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we’re pushing hard on them. Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it’s current shape, Laminar is just few weeks old and developing rapidly, we’d love any feedback or for you to give Laminar a try in your LLM projects!
Tuesday, September 3, 2024
Peace: त्रिपुरा के दो उग्रवादी समूहों के साथ शांति समझौते पर आज होंगे हस्ताक्षर; गृह मंत्री शाह रहेंगे मौजूद
इसमें कहा गया है कि केंद्रीय गृह मंत्री की उपस्थिति में नई दिल्ली में भारत सरकार, त्रिपुरा सरकार व नेशनल लिबरेशन फ्रंट ऑफ त्रिपुरा (एनएलएफटी) और ऑल त्रिपुरा टाइगर फोर्स (एटीटीएफ) के प्रतिनिधियों के बीच एक समझौता ज्ञापन पर हस्ताक्षर किए जाएंगे।
from Latest And Breaking Hindi News Headlines, News In Hindi | अमर उजाला हिंदी न्यूज़ | - Amar Ujala https://ift.tt/Vrx90GF
via IFTTT
from Latest And Breaking Hindi News Headlines, News In Hindi | अमर उजाला हिंदी न्यूज़ | - Amar Ujala https://ift.tt/Vrx90GF
via IFTTT
Monday, September 2, 2024
Health Alert: सामान्य नहीं रही सर्दी-खांसी, फैला नया इन्फ्लूएंजा; मरीजों में मिल रहा एच3एन2 का संक्रमण
Health Alert: सामान्य नहीं रही सर्दी-खांसी, फैला नया इन्फ्लूएंजा; मरीजों में मिल रहा एच3एन2 का संक्रमण Health Alert Influenza H3N2 infection Cough Cold Fever not normal Covid like protocol advised
from Latest And Breaking Hindi News Headlines, News In Hindi | अमर उजाला हिंदी न्यूज़ | - Amar Ujala https://ift.tt/01GlyqA
via IFTTT
from Latest And Breaking Hindi News Headlines, News In Hindi | अमर उजाला हिंदी न्यूज़ | - Amar Ujala https://ift.tt/01GlyqA
via IFTTT
New top story on Hacker News: Does anyone integrate with their customers' DB directly?
Does anyone integrate with their customers' DB directly?
6 by johnyeocx | 9 comments on Hacker News.
Hey all, integrations with third-party SaaS vendors are super common these days, especially with all the AI tools complementing and enriching SaaS platforms, but as the title states, I'm curious to hear if anyone integrates directly with their customers' data store like Supabase, Snowflake, etc. In my last startup, we were building a chargeback management tool for fintechs, and essentially whenever a chargeback came in, we'd only get the transaction ID, so we'd have to query the fintech's database to search for that transaction ID and pull additional data points like name of seller, email of seller, etc. Now, when we did this, it was more of a scrappy workaround to the fintech not being able to dedicate engineering resources to send data to an API endpoint of ours. However, after a while, I thought to myself, there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously, and so now I'm curious to hear how common this practice is. Maybe it's just me, but it feels like integrating with internal DBs could allow a startup to onboard and go live with customers much more quickly. For some reason though, I haven't heard much of this around, so to be completely honest, I'm thinking of starting a tool to make this process easier / standardised. Before that though, I'm trying to wrap my head around the use cases for this so it'd be amazing to hear about your story of integrating directly with a customer's DB -- what was it for and how was the experience!
6 by johnyeocx | 9 comments on Hacker News.
Hey all, integrations with third-party SaaS vendors are super common these days, especially with all the AI tools complementing and enriching SaaS platforms, but as the title states, I'm curious to hear if anyone integrates directly with their customers' data store like Supabase, Snowflake, etc. In my last startup, we were building a chargeback management tool for fintechs, and essentially whenever a chargeback came in, we'd only get the transaction ID, so we'd have to query the fintech's database to search for that transaction ID and pull additional data points like name of seller, email of seller, etc. Now, when we did this, it was more of a scrappy workaround to the fintech not being able to dedicate engineering resources to send data to an API endpoint of ours. However, after a while, I thought to myself, there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously, and so now I'm curious to hear how common this practice is. Maybe it's just me, but it feels like integrating with internal DBs could allow a startup to onboard and go live with customers much more quickly. For some reason though, I haven't heard much of this around, so to be completely honest, I'm thinking of starting a tool to make this process easier / standardised. Before that though, I'm trying to wrap my head around the use cases for this so it'd be amazing to hear about your story of integrating directly with a customer's DB -- what was it for and how was the experience!