Name: [Customer AMA] LangSmith Deployment: From Testing to Production
Uploaded: 2026-04-15T16:16:08.586Z
Duration: 29 min 40 s
Description: [Customer AMA] LangSmith Deployment: From Testing to Production

Transcript for "[Customer AMA] LangSmith Deployment: From Testing to Production": Good morning, good afternoon, wherever you're located. So thanks for joining us. This is our I think it's our third LangSmith customer only, ask me anything, session. And today, we're gonna talk about getting your agents into production, and deploying them arguably one of the harder parts of the the process. I think a lot of you that are here are already doing it, which is awesome. And I know for some of you, it's it's something that you're working towards. So that's what Cory and I are going to talk about today. So let's jump right into it because we're a few minutes late. Wanna be respectful of everybody's time. Cory, you wanna go to the next slide? Okay. Here's what we're gonna do today. Agenda, agenda intros. We'll keep it really quick. What is Lightsmith Deployment? This is what Cory gonna talk about. We'll spend about twenty minutes. He'll go through a demo. The bulk of this session is for questions, though. So any questions you have about what Cory is talking about or anything really LangSmith related, feel free to drop it in the q and a section. And then at the end, we'll talk about some upcoming events and some resources for you. So let's go to the next slide, introduce ourselves. So I'm Lauren. If we haven't met already, I'm one of our account managers here based on the East Coast in The US working with our customers on the East and in EMEA. And I'm joined by Cory. Wanna introduce yourself? Sure. Hey, everybody. My name is Cory Waddingham. I'm a customer engineer here at Linkedin. And, yeah, as as Lauren said, I I work primarily on LangSmith. So I'll be walking through LangSmith deployments and showing how that works and then whatever you wanna know on top of that, this this time is yours. So, Lauren? K. So how this will work? Like I said, as questions come up, please drop them in the q and a. We are recording this session, and we will send it out after the call's wrapped up. We would love your feedback as well. What would you like to see in future sessions? Are there things that we didn't cover that you would like to get to? So please give us that feedback form that we'll link for you all. So that's all the housekeeping. Let's jump into it. Great. So, LangSmith deployment, in a short, compressed version, it lets you run agents in production without building the infrastructure yourself. So there there's three components to this. First, what it is. It's a managed runtime and platform. Why it matters is, gives you reliability when deploying your agents, increases or reduces your time time to production, increases velocity, and helps you scale effortlessly. So, and we're gonna go into how teams use it and what all that means. That'll be the the bulk of the of the demo itself. So let's, let's keep keep moving forward. Alright. So the core problem when deploying agents is that sorry. Hang on a second. My lighting went off. Alright. Sorry about that. So the problem with agents is they break traditional assumptions, around, stateless request response infrastructure. Agents are long running. They're stateful, interruptible, and concurrent. So teams that try to d DIY, the infrastructure for this, they end up rebuilding queues, state persistence, retries, and streaming from scratch every time. So that is the core problem that LangSmith Deployments is trying to solve. So think of LangSmith Deployments as Vercel or Heroku, but for agents. Right. It's purpose built for agentic workloads. It has a managed execution and orchestration layer and allows you to deploy directly from your GitHub repo with very little, configuration required. We we look at everything that LangSmith deployments does. All of the infrastructure that's required to run, run your agents, it's all in one package, all in one place, one thing to manage, nothing else. So what do you get out of the box with this? Well, auto generated API endpoints, task queue is scaling, checkpointing and replay, versioning and rollback, and multi user composition. You have the same APIs whether you're on cloud, cloud SaaS, hybrid, or self hosted. It all works exactly the same way. So there's no lock in on the interface layer. It's all the same same type of control plane. So there are three ways to deploy Lynx with, deployments. First and foremost, cloud SaaS. We handle everything for you. We build everything. We manage everything. There's nothing for you to install, nothing configure. Just simply sign up and and start deploying agents. Then we have two different versions of self hosted options. One is, a separate control plane data plane, where we manage the control plane and you manage the the data plane yourself. This is best for compliance heavy orgs, data residency requirements, working with regulated industries, especially finance and HIPAA. It also supports, multi tenant and per business unit isolation. And then stand alone server, is the the second option for self hosted. Runs within any, Kubernetes or Docker host. The you would handle all the orchestration auto scaling networking. We handle the run time. This is good for teams that already have all the infrastructure in place and don't have to build from scratch. Alright. So who's using LangSmith deployments? There are three types of teams who reach for this. First team is actually building shipping agent systems, AI product teams, shipping copiles and assistance, and internal tooling teams who are running, support bots or support agents. Enterprise call out is reliability, observability, and governance. Those are normally the blockers for enterprise, but with LangSmith deployment, you get all of that taken care of for you. Alright. So let's, let's switch over to the the demo. We're gonna walk you through some of the capabilities of Lynx with the deployments, and then we'll, go on to your questions. So give me one moment. Okay. So this is LYNX with deployments. It's, you you access it on the left hand nav bar. We're gonna work with this travel plan planner agent, today. So this has already been deployed, and there's actually a few different revisions available for it. So we're gonna actually go back. This is the the way the checkpointing works. The reason I'm doing this is I wanna be able to show the difference in two different versions. So let's roll back to this this, older version. It's queued up. It'll take a few seconds. Alright. It's getting ready to deploy. While we're waiting that to finish, just checking to see if there's any questions that come in yet. I don't see any. If you have any questions that come up during the the demo, feel free to pop them into the q and a. We'll try to answer most of them towards the end, but that doesn't mean you have to wait till then to start queuing up your questions. Alright. It's deploying. Okay. Alright. So now that it's deployed, let's go into Studio, and we're gonna create a new thread. So this is where you would walk through and, make sure that your deployment works the way you expect it to. Here, we'll just put in. It'll step through each, each element of the the agent graph, show you exactly what what it's doing, how it's doing it. We can also click through and watch the traces in real time as they come through. This this will allow you to monitor costs on deployment as well as dig into, each each element of the of the trace. Alright. So that's that's run through. This also allows you to go back and view previous runs. No. So you can see what the, the the previous versions, what the results look look like. Alright. What would you say benefits or appointments with deployments are versus AWS? So, Cory, that's kind of a a big question. When we talked about AWS, are you referring specifically to, AWS Bedrock or something like that? Is there a specific AWS service that you're thinking of? Specifically, Bedrock. Okay. So the the benefits of lines with deployments is that you get a, unified interface, not just with, the deployment, but also with traces, with evaluations. If you're used to using LaneChain, as your AI framework, that plugs in directly to LangSmith. So there's really very little configuration to do, to connect the two together. Does does that answer your question? Cool. Excellent. Alright. Let's, let's do this. Let's jump straight to the questions because I think that's gonna be more, more valuable for folks. So who's got their next question? So I can share some questions. You know, similar along the lines to what you just answered, but what's the best way to think about blank split deployment versus rolling your own, infrastructure on something like, yes, or Kubernetes? Yeah. So that that's that's a great question. And the that's the the big problem that LangSmith solves, right, is that, you know, if you're a machine learning scientist or an, a data scientist, you wanna be able to start building your agents. Right? You you want to have a, full infrastructure that handles all that for you without having to add the the additional administrative burden of managing Kubernetes and managing environments and setting up your databases, ensuring your backups are in in place. If your goal is to get to production quickly and to deploy agents reliably, it just makes more sense to use a custom made tool rather than building everything from scratch. Because every every minute or every hour you spend building out or maintaining the infrastructure, that's time you're not building your agents and not testing them and not deploying them. Yep. A lot of people I talk to are already running agents on their own infrastructure. What's typically the point that teams get to when they how do they know it's time to start looking for something more purpose built like links with deployments? Yeah. So that's one of the things that can kind of vary from team to team. Because if if it's a team that already has a a large, infrastructure group and a lot of experience with managing infrastructure, that moment may come later. Right? It may come to a point where they're comfortable deploying all this on their own. But, eventually, you're gonna reach the edge of your knowledge. Right? There's gonna come a moment where the amount of time you're spending just maintaining the infrastructure is greater than the amount of time you're spending deploying agents. That's that's kinda the point where you know it's it's time to have a dedicated resource for this. And for some teams, that that one was gonna come a lot earlier. Right? If you're, again, like, if you're AI, engineer's building out agents and not used to DevOps and SRE work, that moment could be the day you start. Other teams that have large infrastructure IT groups, that moment may come come later, but it will probably come. Yeah. Can you talk us through how checkpointing works in practice? So what does it actually look like when an agent pauses and resumes? Yeah. So as we when we walk through the, that one deployed agent earlier, you see that each step that was running through, that is a checkpoint. The state of the agent at that moment is stored in a database so that you can when you run back through the traces, in the future, you can both see what each point was doing, how long it took, all the steps it ran. And because that state is saved, you can go back, select that actually, let's, let me show you this real quick because it's easier to to show than explain. So down here, you can see let's see. Oops. Alright. So we so at each step, we can change, and add an interrupt before or after. What we want to do is that's not okay. There we go. Okay. So we can change which node we're gonna run this as. We can jump ahead to where it joins the the trip information or down to our starts researching attractions. These are all the checkpoints that are available to us where we can step through and pick up the the run from any of those. So that that is in practice how checkpoint works. Got it. Maybe along those lines. So what happens if something goes wrong after deployment? How do you how do you roll back? Yeah. So if you recall, that was actually the first thing we did, during the, the the demo was we we had a live version, but we rolled back immediately to a previous one. The reason I was the the hope was to to show the difference between the two, but it's taking a little too long. But that is it's as simple as that. You just simply click on the the rollback link and it will automatically roll back to that version. The API endpoints all remain the same, so any application that's using it would just pick right up from there. It's a transparent operation. Got it. Hilka said, love not having to build all the infrastructure ourselves. That's awesome, especially around deployments. Already takes more than enough time to build reliable agents. Totally. Hilka, you mentioned I noticed on the front end side that it looks like there's some native feedback keys support coming. So we have a native feedback integration on the front end since it's already present in the types but doesn't seem fully wired up in ustream yet. In LangSmith deployments, is the idea that this should eventually work end to end with the pre signed feedback tokens slash URLs out of the box and send feedback to LangSmith. The short answer is yes. That is the the intention. Interested as this influence evaluation, testing after going production to production. Totally. Right. Yeah. No. That's, the the ultimate human in the loop is the end user. Right? And being able to get their feedback, and ensure that you can incorporate that in future iterations of the product is very important. One thing that's coming up a lot with customers, especially as we talk about, like, deep agents, is when does it make sense to break something into multiple agents versus keeping it at one? And how do you think about that architecture decision? Yeah. So I I think of this you know, there's a old concept in computer science of primitives, and I think of agents as AI primitives. So the the suggestion, the advice is to have each agent perform a specific task. This is this allows you some flexibility in how you build out future agents, allows you to potentially reuse different sub agents within different graphs by compartmentalizing what they're doing and how they're doing it. This also becomes important later on. Well, our our next AMA in two weeks is on Fleet, and Fleet ties into this very well because it is the orchestration layer, for agents. We deploy them with LSD. We orchestrate them with Fleet. You can also we'll we'll get into more Fleet stuff later, but, but that that is how those two products work together. So keeping agent tied to, as specific task as possible gives you the flexibility to reuse those agents in different ways later on. The the the demo agent we're working with today was a simple travel planner. It had a couple of different sub agents within it. But if this was something where I was building out a larger product, I would have multiple small agents. One that simply does the GeoIP lookup to determine the origin city, one that checks the weather for the dust station city, and then you can mix and match those and change which order they're running in, for different versions of this of the same product. Makes sense. So for a team that's still in prototyping stage, which is a lot of people that we talk to, at what point should they start thinking about deployments rather than just, like, waiting until everything's perfect? At what point in the process should they start thinking about it? Really from day one, for a few reasons. But primarily, because LangSmith deployments comes with Studio as well, it allows you to develop, using the same infrastructure that you you will use to deploy later on. Keeps everything in in one package. It makes a lot easier for teams to share those those runs with others. It's just a and with the the direct integration into GitHub, it allows for CICD of your agents. Right? So as you as you iterate and eventually merge in the main, it automatically deploys for you. This could be a tricky step, getting things from, like, a a notebook or demo environment into production. Are there common, like, mistakes or pitfalls that you see that people can try and avoid? Yeah. The first is trying to use a notebook as a production app. It is honestly surprising how often I I've seen that happen. One of the things that bites people I've seen repeatedly, and this isn't Lanesmith or LangChain specific. This is just trying to translate from notebook or dev environment to prod. It's amazingly how how easy it is to mix up the environment variables being used. That is actually one one of the biggest, mistakes people make when when shifting from dev to prod. And it's it can be very easy to do, especially if you're using notebooks and you're used to using a local dot in file and something instead of something a little more robust. But, yeah, you'd be surprised how often that buys people. Let's play devil's advocate here a little bit. What are the limitations of using LangSmith deployments? What kinds of workloads or teams is it not right for? Yeah. So it's not everything is right for everybody else. Right? Not you had to use the right tool for the right team. And LangSmith deployments is great if you have experience as a a developer, whether it's working in Python or Node or something else, and you're comfortable working with GitHub repos and you're comfortable handling PRs and merges, those are all skills that kind are kind of assumed you're comfortable with, if you're gonna use LangSmith deployments. For less tactical, people who are working on still working on AI agents, Fleet would be a better fit for them because Fleet allows you to work within a web UI, structure to to design and build your regions. Got it. We talk about this a lot, but just to make sure it's clear for everyone on this call, do teams have to use LightGraph in order to deploy using LangSmith? Yeah. No. We we do get that question a lot, and the answer is no. Lynxmit deployments is, framework agnostic. Obviously, you know, BlendGraph is designed to work with it. That doesn't mean that you have to. Great. Those were a lot of questions that we get frequently that we wanted to touch on. If there is nothing else and if Cory you don't have anything, I mean, I'll give people another minute if there's any lingering questions. Again, related to deployments or or otherwise, drop them in the chat. Maybe while you do that while people do that, Cory, do you wanna pull up the the slide deck again? Sure. Just go through a few a few housekeeping things. So some of you maybe will be going to Google Cloud next next week, and then we have, interrupt coming next month. But I'll talk about that in a second. So first, here are some resources, that we'll send. Again, ways to get in touch with us, things like chat LangChain, whenever you have a question about something in the platform, our knowledge base, a lot of resources here for you as LangChain slash LangSmith customers. So check these out when you're when you're working between these calls. And then yeah, the next slide, I think, is what I was mentioning. Interrupt, is our, agent conference next month in San Francisco. So we'd love to have customers there. If you're thinking about it, but you're on the fence, reach out to myself or Carlos or your account manager or AE. We just might be able to get you a a customer discount. So we'd love to have you there. And we have other events that are held by our sort of ambassadors. Oh, but also in our New York City office. So if you are on the East Coast, like I am, we're going to have a session at our office about running agents in production. So sort of a good, segue from this call to to that one. It worked on my laptop. I love that title. Oh, another good time to meet other customers, see how they're building, and talk with our our deployed engineers. So highly recommend. You'll have the link for that. And then we know not everyone is in New York or San Francisco, wherever you're based, if it's India or Seoul. We have a lot of ambassador events, and we're often linking to them in our community Slack channel. So if you're not a part of that, definitely ask one of us, and we'll send you that link, so that you can be updated on events happening in your area. Yeah. So, again, we'd love your feedback. We'd love to connect with you on LinkedIn, stay up to date with what you're building. We share a lot of updates there. I I wanna add Hoka just added, and I'm grateful for his insight here. By the way, from my experience, it's quite essential to deploy as soon as you can to get your organization to learn with the new technology. It's very hard to make Adjantic AI work if it's not inside your company's process in a core part of your organization. Perfecting an agent won't be happening if the people with the know how aren't using it. Yes. I love that insight, and that is a 100% correct. There's no such thing as as perfecting something that nobody's using. Such good advice. And, we know from talking with you all, it can be a little bit of a of an uphill battle to, get the organization on board. The sooner you show them what you're working on and get them working on it, and understanding sort of some of the challenges that come with getting things live, evaluating them, perfecting them, I think the better supported you will be, and, we're here to help you in that journey as well and help you get your your organization on board. So, we'll keep these conversations going, obviously, in one on ones when you chat with myself, Carlos, Cory, other members of the team, but always appreciate the insight from people you guys who are doing it. So thanks so much for sharing. Again, we'll send out a recording of this for anyone who couldn't join. Thank you for bearing with us with some technical difficulties this morning. Happens to all of us. But thanks for joining, and, we look forward to continuing the conversation with you.