This one:...
D\&D - 2025/03/07 12:58 EST – Transcript
Attendees
David Busby, David Busby's Presentation, david raistrick
Transcript
david raistrick: Caddy server. Yeah.
David Busby: have to talk about haroxy and…
David Busby: engine x and traffic caddy. Here we Go. I wonder what this recording is even going to look like when we're done. I don't know if it has a mechanism for gallery view or something. Nope. I don't know what that's going to end up looking That's too bad. Yeah, when we talked about it, you asked just now, does anybody use haroxy? I don't know. what happens if we even look for haroxy?
David Busby: You remember what it was?
david raistrick: I certainly remember…
david raistrick: What it was because I wrote a book on it.
David Busby: What it is? What it is?
david raistrick: So, we didn't publish it.
David Busby: Wait, you wrote a book on it?
david raistrick: We had a fallout with the publisher and…
David Busby: …
david raistrick: that it didn't because when I worked in the video game industry, I had IP requirements that said that anything that I did while I worked for them was owned by them. So, in a gap between working for EA and Zingga, we worked on this and then crap happened with the publisher and I took the new job and that meant it would have gotten wrapped under up under their stuff. So, it's been abandoned since what, 2014. I heard people up there.
David Busby: look, HD Proxy is still having a conference in 2025 in San Francisco, which is sort of some kind of tech hub, and look at these brands that are there.
David Busby: Cloud Cloudflare I've heard of them Microsoft I guess on that's going to be behind Hara Proxy …
david raistrick: I mean, I can see that or I mean Fastly is built on varnish. I don't know the current state but Fastly was 100% varnish based right varnish the reverse proxies no I mean it's it so these all serve the same layer right I mean haroxy was originally pure load balancing but really a load balancer is just a reverse proxy and so everything in that reverse pro proxy market is also just an HTTP server is also just a
David Busby: wait. is I thought proxy was just at TCP UDP only layer, but I'm reading this here and it says quick and H2
David Busby: And so,…
david raistrick: I'm sure that they've added Yeah.
david raistrick: http2 and quick. I'm sure.
David Busby: here's some quick and HTTP3 stuff in there. So, maybe all of my data is super old.
david raistrick: I mean if people are still using it and I'm sure they are somewhere. I just never run into anybody using it anymore. but you're probably seeing it baked into product and…
david raistrick: behind, built into the infrastructure of product. but it's not anybody's at the early stages anymore because you get
David Busby: Look at all these log levels.
David Busby: And it's definitely, check this out.
David Busby: It's definitely HTTP aware. I didn't know that at the time. They've got this other stuff here where they're like, "Hey, you can fetch out interesting stuff at higher level layers." OSI model s***. Remember that?
david raistrick: Yeah. Yeah.
david raistrick: Yeah. Mhm.
David Busby: is how to fetch stuff out from HTTP layer 7 even these days. Big deal. so that's actually a lot that's cooler than I never got heavy into haroxy.
david raistrick: Yeah, the big problem 10 12 years ago was that nothing had integrated HTTPS support.
David Busby: I was like, it's a thing." varnish was HTTP only,…
david raistrick: So not nothing. Engine X had integrated HTTPS support but HTTP only right.
David Busby: You always had to terminator in front.
david raistrick: That was yeah you had to run S tunnel terminator Apache you had to put something in front of it if you wanted to terminate SSL. and in fact we would do that with ELBs. So ELB would terminate our SSL even though ELB didn't have SSL originally but once it did that saved us a lot of overhead managed that distributed out to varnish or to haroxy in one platform we used varnish and haroxy both because primar we used it as an SQL load balancer. Yes, you can do that.
david raistrick: And we used it for Blaze which was a binary GP G what's the RPC communication protocol right RPC so it spoke heat was the binary version was an XML over HTTP so…
David Busby: Take your time.
David Busby: f* You mean their elastic load balancer?
00:05:00
david raistrick: but we had to support both of those depending on the clients connecting And so being able to do that load balancing for that was which today we can do that in AWS we do that with an ALB right because ALBs or you do it with an NLB right to do the binary protocols but yeah so in the elastic load balancer they have all these different pieces the network load balancer would do the
David Busby: when I started in this game a cajillion years ago, it wasn't that long ago, but it was still in the 1990s. and I was at Microsoft. I worked on a component of NT4 Service Pack 6 that was called WLBS, the Windows load balancing service, and it basically spoofed every machine in the cluster had the same IP. And so it was multiccasting packets to however many hosts you had on that cluster IP.
David Busby: they had a cluster IP, and then you would just blast packets at all of them. So, you blast your packets out at up to 32 nodes in the cluster. 31 of them just say that's not for me and drop the packet straight away and then one of them does its thing, It was very good for, things like if you needed the FTP load balance or if you needed the HTTP load balance,…
David Busby: it was great for services like that.
David Busby: there was a different solution if you were trying to load balance Microsoft's SQL servers right they used that what was it at the time it's called wolfpack so I'm so old nowadays nobody has to deal not nobody but nowadays very few people have to deal with this until you're very big right you get into Amazon you check a box you get elastic load balancer right is it haroxy maybe is it some custom stuff they've could be HA with some custom stuff and you would hardly know, Check the box. Everything seems to work just fine. So was that. And you were also talking about so you can't use engine X necessarily for high availability because it's just one box. You don't engine
david raistrick: Wait. Yep.
david raistrick: No, it has load balancer functionality too. So you can when it's one box.
David Busby: but there's a box, isn't there?
david raistrick: All of these are one box.
David Busby: I thought proxy was able to do that and…
david raistrick: In order to do anything else you still have to load balance in front of them in some way. So to make a highly available that is the functionality of so originally that was VERP right VRP or…
David Busby: basically put itself two whole side by side and fail over or even Heat.
david raistrick: or it was HSRP but HSRP was owned by Cisco so it became VERP and then VERP became something else. So this is something that you can do outside of AWS. So when you're operating inside your own network infrastructure that supports multiccast, that's how you do that. if y it 10 years ago haroxy did not have this built-in support. You still had to build that layer on top of it. All harroxy is I run it through a selection process and say this request should go here here based on whatever rules I've got in place.
david raistrick: you still have to have distribute those boxes and get traffic distributed to those boxes. the cheap way if you weren't doing VERP was to and I know that Linux kernels have evolved to have some other support for something VERP today. They've changed the name. I believe there was a bunch of legal issues continued legal issues. but basically…
david raistrick: what you're doing is you're saying you're presenting multiple machines as a single IP address to the network, And what VERP would do is there was a heartbeat protocol over multiccast that would say that would communicate over multiccast to all the machines listening to that multiccast address that would say, "Hey, I'm the boss." Exactly.
David Busby: Yeah, sim that's similar to…
David Busby: how the Windows load balancer works.
David Busby: And there's a registry config for how frequently you want to have that heartbeat go, right?
david raistrick: Yep. …
david raistrick: it didn't handle load distribution to your load balancers. So, if you had two load balancers running VERP or similar type of protocol, then your traffic is only going to one of them. But if the other stops receiving its heartbeat, it takes over. Now it does this by assigning it that IP address there.
david raistrick: It creates a Virtual address. Exactly. Yep.
David Busby: Yeah, so you're moving the IP from box A that's failed and…
David Busby: moving some IP that existed there to a different one possibly doing a little ARP cache flush or something in your local Ethernet side of the universe.
00:10:00
david raistrick: Yep.
David Busby: But if you were fancier for some definition of fancier you could have haroxy. Let's just pick on them with some kind of similar Windows setup to that multiccast thing.
david raistrick: Yep.
David Busby: They both have their multiccast the cluster IP and incoming traffic, hits your edge router which maybe still, now you have to fail over that thing with other mechanisms which is out of scope for today. But that comes into your multiccast network.
David Busby: You throw that same packet at two different hosts in number of hosts. one of them decides this is garbage for me. The other one decides according to some hasht rule or something, I like this one and…
david raistrick: right?
David Busby: I know the other person doesn't like this one. and then it just passes through now to your backend application servers which could be seven or eight different servers, on the back end. And it looks like, and earlier, I didn't know this 5 minutes ago, but Harroxy with its layer 7 inspection, you could also use it to have your single fancy end point on the internet, adosio.com.
David Busby: and adosio.com resolves to some IP address that's on a network that I control behind my fancy edge router and I've set that up to be multiccast to haroxy 1 and haroxy 2 and then they both have really clever rules on them so that when you hit idosio.com something clever over here it routes to backend application server one, two, and three. And if you hit adosio.com/fu, it routes to these three over here. And if you do adosshio.comws, it knows that that's probably a websocket connection or probably is smart enough to do header inspection. So you don't even have to make that choice and it's like a sub route. you went to adosio.comfu and then you tried to do the, upgrade to websocket.
David Busby: Ajoxy just has some inspection rule because I don't know enough about caddy to know if you can do that. I just now am thinking harroxy the one I had used before.
David Busby: You can do that. I was looking at these Engine X is definitely setting itself up as I'm a single box at the front and I'm just doing SSL termination very likely and then I'm routing this on the trust semi-rusted backend network to hit app server one two three and four. Do you
david raistrick: Yeah. Yeah.
david raistrick: And it's more heavily used these days. Engine X is le we see it less being a load balancer and more as just a proxy server to a local less featureful. So we'll see it in front.
David Busby: Whatever that one is.
david raistrick: We see a machine. This might be a Docker container running EngineX with say UWSG behind it or UW SDGI, right? the Python integrated web server,…
David Busby: I know what you're talking about though.
david raistrick: the one that everyone uses that's actually scalable. that one,…
david raistrick: but it's not a fullfeatured web server, right? It speaks HTTP, but there's not a lot of rules.
David Busby: Yeah, speaks HTTP sort of.
david raistrick: Exactly. It speaks HTTP enough, but you don't want to put that directly in front of your customers or more importantly in front of your adversaries.
David Busby: Here it is.
david raistrick: And UWSGI,…
David Busby: It's UWSGI.
david raistrick: that's the one.
David Busby: Setting up for load engine X here saying they can do the load balancer for fast.
David Busby: UWSGI, SCGI says you can do mimcache,…
David Busby: gRPC. I know that EngineX was also having some features to do mail proxy, which I thought was a weird feature set. you dig around in their config and you'll see that there's something mail config.
david raistrick: possibly. Yeah,…
david raistrick: I don't remember if anything, but yeah.
David Busby: Never used it, I've only ever used engine X t the things I would do typically, and I'm at smaller scale so I rarely have to worry about or when I'm in that zone I'm already in AWS I'm already in Google Cloud and I point my IP to whatever load balancer I've constructed in their universe and I'm sort of fingers crossed I check the necessary boxes here for availability zones etc.
00:15:00
David Busby: I'm hoping that my DNS integration is good for that. I'm hoping their multiccast is good. I'm hoping their configs work. and it generally does most of the time. But then, when we're hosting in there and we want to have reasonable performance, but we still want things to be, air quotes bulletproof, right? You put engine X on the front, give it some caching so that all that so it's load balancer, then it's engine X and with a generous cache. So any of the static assets that get served from the backend can now just stay pretty hot in engine X.
David Busby: And then when you do your app requests those get load balanced to some backend which is in some of those cases the engine X on the front end and it looks like a single user experience to the client but one of the backend requests is landing on Apache with PHP and the other one's landing on a Python and there's this one little component that we've got…
David Busby: where the websocket is involved that routes to NodeJS thing, but then there was this other special websockety spot where the node wasn't performant. So, we had to do this other one special thing in Golang.
david raistrick: right? Yep.
David Busby: And now you've got four or five different backend services on this what they used to call the DMZ, right? Remember that one?
David Busby: And then ELB or something.
David Busby: Engine X is now really just sort of like my app router. And I do not think that I find the EngineX config file a little arcane.
david raistrick: It's also extremely limited.
david raistrick: If you ever look up the religious arguments around using an if statement.
David Busby: Don't use because if is go to, right? Some velociraptors are going to come and get your ass, dude. No,…
david raistrick: So the…
David Busby: that's for I've read it twice because I myself needing to do that.
david raistrick: if you have that if you have the need to use an if statement stop pick up open resty. So open restyi is engine x with a Lua langu Lua plugin and now you can write using Lua you can create your rule sets and make your actual if statements and all the things because you have a full programming language at your disposal instead of their extremely limited configuration language. when we used varnish back in the day for this type of thing…
david raistrick: because we needed to have intelligent decisions happening in our configuration structures, right? and…
David Busby: Yeah. Yeah.
david raistrick: EngineX didn't allow, they had this thing called an if statement that was not what you think it was. And so you couldn't use conditionals or you couldn't use conditionals in the way we expected to. later when open resty came around that I mean open rest is really just a series of plugins wrapped around you can do engine x and plug in the lua plugin just fine but open rest is the easy path to set it up.
David Busby: Yeah. I don't know.
david raistrick: And now everything you might want so in varnish we had an inline C lang. So we could use C in the actual decision logic right using inline C and of course it's C good and bad it's C. It was awesome because it meant that we could go and have really complex programming logic that would happen. we would even fork our back-end application identify high constraint high high traffic high load pieces of it that is in that these are back in Java or…
David Busby: Yeah. Is it forward?
david raistrick: these are in PHP and pull those back rewrite them and run them C in varnish. So now we pushed that all the way back to the load bal or well to the proxy. Yeah, we moved it forward and…
David Busby: Yeah. Yeah,…
david raistrick: it was a lot lighter because we were able to strip out a bunch of stuff and write it in pure C. and these were things that were very highly active. One of the reasons that we brought these tools in the reason we would bring Varnish and…
David Busby: the hot path optimiz
david raistrick: EngineX in at all to begin with was the big win was for that caching layer, and that's so stat assets, but not just static assets, we would cache API responses.
David Busby: I mean,…
David Busby: I'm looking at some of these plugins right here for Varnish. Seems like you could write your own pretty quick.
david raistrick: Yeah.
David Busby: And there's loads of these examples to pick up four or five for picking up, IP goip crap, hash IDs, move some bars around,…
00:20:00
David Busby: JSON parsing. All right. Where's the one that's integrating email? no application is complete until it's integrating with email, Probably down here somewhere. Points to Reddus. what is that for?
David Busby: this is to put varnish cache in front of Reddus. That's about Yeah.
david raistrick: This is to allow you to use what's in your Reddus database or…
david raistrick: use Reddus as a database for accessing, making configuration decisions, making routing decisions, making whatever if you're building this configuration layer. So, think about Fastly, Cloudflare, what they're doing is they're receiving a request for hundreds of thousands of different possible websites into one single cluster,…
david raistrick: So, they need the ability to look at the simple version of it. Let's look at that request header and it's catty.com right and…
David Busby: Trying to upgrade the website.
david raistrick: it says where does this go? We can't compile all of that into one little static config file. So we need to look somewhere for that information. So we'll use the Reddus plugin to ask Reddis say hey I make this database query I have that what servers are currently active right that's the type of thing that you would use that for
David Busby: Yeah. Subnets and Sentinels and other interesting stuff that you get to pull out of I'm looking at the docs for one of these.
david raistrick: Yeah. Yeah.
David Busby: LIB vmod reddus. This is just like the docs of what it does.
david raistrick: We did 20 or 30 internal Vmods for our game platforms.
David Busby: So now there's another choice.
David Busby: There was at the start HA and EngineX and Caddy and Traffic and we haven't even gotten to Caddy and Traffic like the new dogs. And here we are. HA proxy still alive and viable and has a conference coming. That's Engine X is still a good LB, but it's not the multiccast accepting front end. Engine X won't do that. Haroxy might be able to do that. Then you go to EngineX to then route to your application. Open resty is just a better extended engine X.
david raistrick: If you need logic.
David Busby: That's what it says here.
David Busby: It's extending checks and…
david raistrick: Yes. It gives you right.
David Busby: has l of libraries and that makes it easier than the sea level vod and now you're the races right that's their value a little bit heavy for some of that stuff and…
david raistrick: And writing in open resty is so much easier than writing in C. Yep.
David Busby: the general generally the app devs
David Busby: working in a language that is not C and then getting stuck writing C that is also now inside of some other bigish platform thing with all of the interesting C tricks that they've done I've not done a mod for Varnish but I've done one for PostgreSQL in C and you have to work in the PG's ecosystem and they have these
David Busby: magic functions that you've got to do with this magic stuff. and I don't use C every day. So, it's like, let's remember C. And fortunately, a lot of the stuff from 20 years ago is the same swish. But understood. So, then now it's 2025 and you have a green field project and you've got harox assume that just throw it in Amazon and Google Cloud or Azure is out. But now you're making your own choice for your own. You just got your own rack with your edge router at the top. Maybe you've even got access to BGP. So you're like the full network god, and you get to put what are we going to do right after our edge router Would you do traffic? Would you do caddy?
David Busby: Would And I'll just open some pages here. I finally turned the screen sharing on. We were talking about this stuff and I thought I was sharing my screen but I wasn't dumb ass. So that's sorted now. T R So does traffic.
david raistrick: I would do caddy.
david raistrick: If so on the front of it where I'm terminating SSL, I would do Caddy because Caddy has very tightly integrated Let's Encrypt support. Does it?
David Busby: Both of those will do the,…
david raistrick: Okay. Right.
David Busby: Acme fetch me a certificate. It kind of easier than it will for EngineX or…
00:25:00
david raistrick: And it's fully integrated for Engine X,…
David Busby: Apache. Yep.
david raistrick: you have to bolt in external scripts that run and then you have to actually restart EngineX to pick up the changes, Because it has to rewrite the engine X configs and engine X has to restart. There's no way to even do a hot reload on Caddy, it's fully integrated. It's literally a single line of config to set up your let's encrypt stuff and Mhm.
David Busby: But is Caddy gonna give me my So the thing I'm looking at traffic for is because I want the smarter app layer router which is for me SSL terminate and then the routing rules that I got to construct in engine X is they work. I wish I could make them smarter and a little bit easier for me to understand. I found traffic because lots of people using the Docker universe point to it. You can see loads of these Docker Compose files have one or more things going on, one or more services in there, but the top level service of it is like, we have traffic running.
David Busby: And then there's these very complicated I'm sort of over my own head with it sometimes with the traffic config.
David Busby: you can make these magical labels in the Docker universe and then somehow those magical labels on this service tell the traffic service that thing is available with this router and…
david raistrick: It's just internal DNA.
david raistrick: If you look at the labels is just internal DNS, right? It's inside that split horizon DNS.
David Busby: yeah that's cool and all but then sometimes it tells you p.outer router something and sometimes you're is the required name or is one of these my nice little variable that I could pick my own name on and then have to remember to be consistent throughout the rest of the use in my docker file I don't always follow that and…
david raistrick: I haven't looked at traffic in a long time. When I did, it seemed overly complicated and cumbersome in the config. the kind of can do your basic reverse proxy functionality.
David Busby: then caddy but caddyy can do the routing
david raistrick: Basically all the same stuff you do with engine X,…
David Busby: That's all I'm ever doing.
david raistrick: "Let me look at a host header," or, "Let me look at a port that I came on in on," whatever.
David Busby: I look at the host header to route to the back end, right? Because app do something versus API something goes to,…
david raistrick: Yep. Or look at the URL.
David Busby: this. And I want to look at part of the path…
david raistrick: So I take,
David Busby: because adcomfu goes to the foo handler of the app which is our legacy codebase that's running on this old crusty infrastructure that we're working on getting out but it's not out yet. And then if you go to app.adosio.combar
David Busby: adosio.com/bar. that's the new feature set, the updated UI or whatever nonsense and that needs to route to the other app server because we have a totally new app stack that's running that because our legacy stuff. We didn't want to keep grinding it forward, That tech debt is too much. Let's throw it away when we can, but at the same time, let's have a front end that still exists, the sort of less contrived example right app.adosio.comv1 goes to the prototype code api.adosio.comv2
David Busby: adidosio.com/v2 goes to the better one and we can just route at the front end with caddy or traffic or engine x or…
David Busby: whatever you think I haven't seen it but maybe this is just momentum that is making that happen I haven't seen a lot of caddy in the container suite Yeah.
david raistrick: I mostly ever see EngineX in the container suite.
david raistrick: So it's interesting that you're seeing traffic.
David Busby: So these are a container suite where I'll pick on my side project openc that when you run that you don't just run app.openc
David Busby: Open THC it's not like microervices per se…
david raistrick: they're premises.
David Busby: but you need to connect to some kind of compliance reporting engine which is there and then there is other parts of the project that are intended to be well separate from the primary application there's front end so when you run it locally for yourself you've got app dot you've got your compliance engine that's doing all of this logs and compliance. And then you've got a POSOS And then the POSOS front end inside your shop, you might want to do one thing. If it's exposed to the internet, when it's like a delivery driver, they want to do something else. And we've been working on the Docker suite with traffic at the front. And for the host names, it's just routing it straight into the container with the same name.
00:30:00
David Busby: POSOS, this goes to this, and then you can say, when you're coming in from somewhere else, this path is restricted if they have their thing on the front end or in some cases, there's an even weirder setup where you have a server on the internet VPN into your office and the people on the internet delivery drivers and stuff, they're visiting, poss.comp.comdely and that's hitting some remote server way over there that just goes through the little VPN tunnel to the internal box, and that's not really traffic that's handling that except that I suppose you could set up some route like that,…
david raistrick: Gotcha.
David Busby: but that's where I said, " I have multiple hosts in my Docker. I need to find something."
David Busby: and the internet sort of routed me to just use traffic. but if Caddy is doing that and I see caddy file request matchers client IP header reax path query protocol probably lets me it does stuff with HTTP or gRPC and HTTP2. That's pretty handy. named master wild card do something with the thing the only one I'm trying to figure out is here it is says common patterns doing something with PHP static file spas trailing slashfix that's kind of a fun the only thing I'm looking for is upscaling the request so it knows if you're a websocket to send it somewhere
David Busby: They're not funny.
david raistrick: I don't know.
david raistrick: I mean, I assume it's got some websocket support, but websockets are a funny thing. if…
David Busby: They're awesome.
david raistrick: if by awesome you mean completely breaking your scaling model…
David Busby: What are you talking about? No, that's…
david raistrick: because now we go from…
David Busby: why I got multiccast load balancer in the front end. We were just talking about this, multi in the front end and…
david raistrick: but only but…
David Busby: infinity number of app servers behind it.
david raistrick: but what you've done is you've locked a client to an app server. Right. Exactly.
David Busby: No, no, no. Only until my websocket fails,…
david raistrick: Exactly. that's only problem solved…
David Busby: which fortunately we don't have to worry about that happen because we know that's gonna happen. And then they'll just Problem solved.
david raistrick: if the reason that you were implementing websockets was because you needed the persistent connection so that you had live data stored in cache on that app server.
david raistrick: Right? a lot of times the reason that we implement websockets is because we wish we had persistent connections not because Yep.
David Busby: I don't know about this persistent connection…
David Busby: because before websockets you could do server sent events SSE they would say do some SSE and SSE kind of sucked because you'd have the open client connection it's a regular old HTTP connection so it is browser limitations built in on but how many open connections you got and you have the SSC and then you do have the long polling is happening on your server, right?
David Busby: the client request comes in through your magic load balancer and…
david raistrick: Yep.
David Busby: all that sauce and then lands on your app server finally which is subscribe waiting on a event to get published on Reddus and there's some kind of like spin lock right check the status of something if the file doesn't exist or whatever crap you're looking up DB reddis file whatever doesn't exist why don't we sleep for one second feels good to me, And then check my thing again and then check my thing again. And if my SSE loop goes for more than 60 tries, push back to the client, hey, I found nothing this time. And then your client needs to be smart enough. that session, for lack of better term, did nothing. Let's reopen it.
David Busby: I think there was another funny term that folks used before. Ajax was the common term for stuff like this.
david raistrick: I mean really…
david raistrick: what a lot you would use HTTP.
David Busby: There was something for blocking long polling requests though.
david raistrick: Because I mean using an HTTP keep alive keeps the socket open but available as long as everything allows the keepives to come through. So you've made a request it's a keep alive supporting the load balancers and all the path along through in the web server all allow keeps and so that's maintaining a socket. So your next request is still a request, but it didn't have to open a connection.
00:35:00
david raistrick: But it is all still polling, right? because the expensive part of a request you're is opening the connection, making the request,…
David Busby: necessarily. You've got to do that too.
david raistrick: receiving the response, And so when you make a lot of small requests Is there data Then if you had to keep reopening the connection that becomes a lot more expensive.
David Busby: All right,…
david raistrick: The nice thing with keepives is that it's easier on the load balancers than websockets. if you scale that up to say 100,000 concurrent users
David Busby: right because keep alive the sockets's like a semi-open state or something like this…
david raistrick: Right. It is a persistent active connection.
David Busby: if I remember correctly right and then the
David Busby: Websocket is definitely always on, It's always always on.
david raistrick: Right. Right. Yep.
David Busby: I was just doing side note here. I was just doing some digging path for websocket on your routes. You would just say handle slash something with some rules here in caddy. And here's some documentation. It says you put at symbol and looks like some kind of pattern matcher or something which I don't know enough about caddy,…
david raistrick: And …
David Busby: but it seems like
david raistrick: that's your upgrade to websocket header that it's adding,…
David Busby: That would be I mean this says connection upgrade websocket and…
david raistrick: right? That's telling that sounds right.
David Busby: I think this is like a matcher that says if the header does this then you're at websocket rule again I don't know the whole details at symbol seems like something special maybe a reusable rule it's like that in other things you say handle path and…
david raistrick: Yep. because your client has to support websockets.
David Busby: then you say and…
david raistrick: Your requesting client has to support websockets.
David Busby: I'm websocket then do this next little group.
david raistrick: So if it sends and says, …
David Busby: Yeah. Yeah.
david raistrick: if it sends the connection header that says I'm allowed to do websockets and then you support it, then you correct them too. But the thing is…
David Busby: You're here showing us like you do a name matcher header connection upgrade and…
david raistrick: though when we
David Busby: you've done that and here's another one from there cel expression must be something caddyy special but here you have it and then you get this and then you would just say at websockets and…
David Busby: then now my list of back-end servers is my NodeJS or my Golang one because I have this other app that's Ruby or Python or PHP or cold fusion perhaps things of that nature ASP 3.0
David Busby: Which now classic,…
david raistrick: It changes the scaling.
david raistrick: So part of the reason that we use these reverse proxies instead of just going directly to our application server HTTP endpoints is to decouple one of the things that we do is we decouple keeps.
David Busby: right? Right. Yeah.
david raistrick: So The load balancer will keep alive to the application server because we only have a limited number of sockets that we can ever use on a machine. we'll just round number this and call it There's Many of those 64,000 sockets are in use. But if your load balancer has it's got a socket to the client and…
David Busby: All right.
david raistrick: a socket to the server. So now we're down to 30,000, right? Maximum connections because it's got to do both because it's one armed and it's got to communicate to the client, it's got to communicate to the server. Now if we have a 100,000 active concurrent users who are doing polling requests where they make a request pause for just a second right and another another client makes a polling request pauses for a second we have a fairly scattered right sca we've spread that load not well controlled…
david raistrick: but we've spread that load across a 60-second interval. So the connection load,…
David Busby: Yeah. Hey.
david raistrick: those 100,000 connections are spread across that 60-cond interval. If we have a 100,000 websockets, we have 100,000 sockets concurrent 100. We have 200,000 sockets open concurrently on that load balancer, which means we now have to be able to pull a load balancer out who's at capacity of sockets. One of the reasons we use load balancers like this are because they're generally more performant than our application server and…
00:40:00
David Busby: They're only doing that one thing, right?
david raistrick: they're only doing that one thing,…
David Busby: I mean, there's the other nice thing that happens when you put them in front and…
david raistrick: right? right?
David Busby: you put a fail to ban or one of those other rule sets on there is that all the b** traffic on the internet dies on a machine that's really just designed to open and close sockets with a very small set of rules, right? So, all of the horsepower can be dedicated to just
david raistrick: Because a simple connect and reject to your load balancer. You can tune the TCP stack on your load balancer to be more efficient for load balancing purposes. And on your app server you would tune it differently. the actual application that you launch does things differently. So your load balancer everything in this class of software can reject a request connect and…
david raistrick: reject a request in let's just call it five milliseconds right when your application server your heavy stack you connect to rest to jetty with java you're looking at 50 milliseconds probably
David Busby: Right. …
David Busby: I gotta also when I connect to that back end because this is my legacy app that's called Fusion or PHP or Ruby, that connection needs to then open a database connection to something else behind it.
david raistrick: right? and…
David Busby: And now it's not milliseconds, it's 200.
david raistrick: this is just to execute the logic of should I have is this request even something I was supposed to answer.
David Busby: I got to look up the guy's authentication status.
david raistrick: Right. So 404 versus a 404.
David Busby: There is a reason I was picking on cold fusion.
david raistrick:
david raistrick: Right. And then that ramps up. So he's meant to call fusion one of the platforms that you license per connection. So how many connections are you licensed for? and the default configuration at least it used to be was 10 threads. it could only ever respond to 10 concurrent requests at a time. That was the default configuration that most Cold Fusion operated at.
david raistrick: So if you needed those 100 thousand users you needed a lot more right so by decoupling that and making that you mean you decouple and you move that polar right you separate that polling service out to its own service that's much lighter it's running in anything other than cold fusion even though your core app stack is in cold fusion you say okay I've got my polar that says is this available and now you do that to your websockets too you connect your websock to go or whatever …
david raistrick: but you do maintain that persistent connection and you have scaling overhead from that. it means your load balancer layer becomes larger, right?
David Busby: Yeah. I'm disappointed that this caddy wiki doesn't have a picture right on traffic.
David Busby: You look on traffic and you get a nice picture right at the front that tells me all the stuff I wanted to do. API.me.com me.com weird s***. and yet another subdomain. So, I'm routing by path, I'm routing by subdomain. It shows me the little picture of the HTTPS termination and is telling me nice things about it, too. Dynamic routing, auto discovery for new stuff that's on the back end. I don't know if that's in Caddy or not. Probably, right?
David Busby: Lots of these things are all just, converging at common feature sets and…
David Busby: then one of them has it and you're like, " my current one doesn't have it this week, and then next week someone's like,…
david raistrick: Right. Yeah.
David Busby: "Hey, I wrote a new mod that tells you how to do dynamic route or something and it gives you metrics, tracing, and logs." that's it, And I guess they're saying, "Look, we're an application proxy, which really maybe means they're a HTTP and similar
david raistrick: Right. it can serve static assets.
David Busby: proxy, but Cadd's also that. but traffic isn't a web server at all. And Caddy also traffic does.
david raistrick: It can serve static assets. It can serve a tree of files out of a directory. Yeah, I am sure.
David Busby: I don't know about that. It always is. I've never seen an example with that one.
David Busby: and it says configuration discovery I don't know frequently asked questions I mean that's the next thing there caddy file reverse proxy something something there's got to be a mod that boom told you so I just love seeing it so I had to do it yeah traffic I think is just this routing thing and…
david raistrick: Does it have a caching layer?
david raistrick: I don't think Caddy has any kind of significant caching layer either, but no. Okay, I was wrong. Traffic does not serve static files at all. Yeah.
00:45:00
David Busby: I don't think it does or at least I've not seen it doing any kind of caching stuff either.
David Busby: Maybe you have something like just operating at the TCP layer HA proxy getting the multiccast throwing that away. Then your haroxy lands to your traffic slash caddy app routers which then may route some stuff through varnish right and keep all my hot static assets in varnish and then I have my app servers behind that. So, we've now drawn a fairly complicated thing.
David Busby: Does I Wonder if we can get a diagram of that on Mermaid? You ever use mermaid? Man, I love this damn.
david raistrick: I have GPT create memory diagrams a lot.
david raistrick: Yes. Yes. No.
David Busby: You make GPT do it? You don't like to just overthink it No. Sad that. I don't know.
David Busby: Let's ask I'm not even going to spell check my s Draw me a mermaid diagram. ha proxy multi-cast front. How about passing to traffic app router which pass which passes to which proxies let's make sure we're using good words which proxies to four app servers a varnish varnish cache Apache
David Busby: plus PHP. what other dumb thing?
david raistrick: I would let it figure it out.
David Busby: NodeJS websocket. We soocket and go Golang, HTTP and websocket server. What would this be? The entity relationship diagram or…
david raistrick: That's…
David Busby: diagram? Let's just say a diagram.
david raistrick: what I would do.
David Busby: Give me a f** diagram.
david raistrick: You gotta tell it to write the code for mermaid live. Or wait, what's it doing? it's generating diagram.
David Busby: It's generating a diagram. I'll structure baby.
david raistrick: I think they may have added in mermaid support at some point recently.
David Busby: How could you not? I mean, it's right there.
david raistrick: No, that's Dolly for you.
David Busby: Man, that is actually a very cool diagram.
david raistrick:
david raistrick: It is.
David Busby: like zoom in and…
david raistrick: I think that is a strategy.
David Busby: enhance when this is on the YouTube this one actually looks cool. What's this thing over here? it says it's a tragedy. it's my operating server and…
david raistrick: It's an up river.
David Busby: it's a tragedy over here. sometimes there's these frontend systems behind something in the back end that is a tragedy.
david raistrick: I like your subwoofer stack over there though. but tell it to write code for live.
David Busby: Yeah. All the sound, baby. …
david raistrick: Yeah, there you go.
David Busby: man, that is hot. Let's see what that looks like on Mermaid. That's pretty cool.
David Busby: Except that really I need to have haroxy one and we still have to have another top level thing.
david raistrick: Do it.
david raistrick: Tell your intern.
David Busby: Tell my intern.
david raistrick: Don't fix it yourself.
David Busby: What do we do here?
david raistrick: Tell your
David Busby: We say no no no no. The HA proxies are behind an edge router. in number of traffic spelling and then the four infrastructure F r a s r systems subgraph.
David Busby: What the hell is this? Proxies to weird ass names. One of the things I was just telling folks is this thing comes up with some weird names for your variables that actually looks like how I feel like that should look.
david raistrick: that has got some relationships.
00:50:00
david raistrick: It's probably not wrong. I mean, it's, Yeah, there you go.
David Busby: I have my multiccast edge router and that hits me to my haroxy one and haroxy 2 and then that routes me to my backend split, right? You can see haroxy one and two both will take their connection to one, traffic 2 or traffic in that was very nice. It knows that we could possibly go more and then I do not know how else you would have it without hell of lines connecting three traffics to four different backends.
david raistrick: Ask it to propose a full production architecture for this as a mermaid diagram.
David Busby: You're going to have lines crossing each other all over the place. I wish you could make it do a straight line. This isn't it?
david raistrick: It's a star. I don't know. Ask your intern.
David Busby: What do I ask it?
david raistrick: Say you want to make this production. What are we missing? Ask it what we're missing.
David Busby: This is good. What's missing before we could put this arc seeds here in production? Before you should consider this s*** ensure multiple …
david raistrick: Look at that. We're down here.
David Busby: yeah, that's for autoscaling, right?
david raistrick: We're just building network architecture. Come on now.
David Busby: But this is for your proxy should be an active. I mean that wasn't explicit. we didn't say that…
david raistrick: Keep alive. That's the Chad GP even called out verb or…
David Busby: but that is what's shown in the diagram. At least I thought it was.
david raistrick: keep alive D.
David Busby: Yeah. use a firewall and then it suggests Cloudflare or something like that. Not what I would choose. Put some logging Prometheus and Graphfana. all the stuff, ensure varnish cache is tuned. Yep. It's always been a spot where you've got to tune it or it doesn't work like you want it to. Ensure that Go and Node can handle a lot of traffic.
David Busby: Maybe start using a message Gave me told me rabbit.
david raistrick: right?
David Busby: Wish it would have just I wonder why it suggests specific things, Here it says implement failover and then it tells me use Slack or pedagra duty. it's routing me to brands and I wonder man I guess there's some brands because you don't have use Slack. You could have Discord. You could have the new one I saw yesterday, Revolt, or my personal favorite, Mattermost, and they're not even mentioned here. That's unfortunate. Her duties in there, but what about sending the SMS directly through Twilio on your own? Not mentioned. Here's how to do some CI/CD. you got to have that.
David Busby: I mean, that's all cool, but I feel like that's GPT getting a little bit out of scope for is this here.
david raistrick: Yeah. But you do know…
David Busby: But definitely your multiccast edge router, we've moved the s single point of failure all in the diagram all the way to the multiccast edge router, but there's another problem there that's probably not for the site reliability team and it's probably not for your dev ops team. It's for your net ops team.
david raistrick: because your goal here is to build a resilient architecture. You need to communicate with whoever is working on that and understand what they have what their limitations are, right? how that their failover works because their failover needs to match your failover options,…
david raistrick: and once you build this, you should start pulling plugs.
David Busby: This is yeah,…
David Busby: this is nice, although to get a lot of stuff like this, I can see why folks just slide into AWS or GCP and let me just check a couple of boxes and move this slider up to high availability, freaking done. Anyways, this was a great jam session. I think, this was our number one of one. I'm going to probably just post it as is online.
david raistrick: Sure, let's do it.
David Busby: I don't know. I was trying to get my Apple account set up so I could get something there for podcasting, but I did not even have an Apple account in 2025. And then when I was trying to sign up, it was like, we can't text your phone number. Maybe Apple and Mobile are mad at each other. I don't know. I'll try that again and we'll see what that turns into.
David Busby: and maybe we'll try this again Mondayish or something. So appreciate it, This was cool. Where is Sorry, I got to press some other button up here. All right.
Meeting ended after 00:55:02 👋
This editable transcript was computer generated and might contain errors. People can also change the text after it was created.
|Routes to| HA1
ER -->|Routes to| HA2
HA1 -->|Load Balances to| T1
HA1 -->|Load Balances to| T2
HA1 -->|Load Balances to| Tn
HA2 -->|Load Balances to| T1
HA2 -->|Load Balances to| T2
HA2 -->|Load Balances to| Tn
T1 -->|Proxies to| V
T1 -->|Proxies to| AP
T1 -->|Proxies to| NJS
T1 -->|Proxies to| GO
T2 -->|Proxies to| V
T2 -->|Proxies to| AP
T2 -->|Proxies to| NJS
T2 -->|Proxies to| GO
Tn -->|Proxies to| V
Tn -->|Proxies to| AP
Tn -->|Proxies to| NJS
Tn -->|Proxies to| GO
TEXT;
echo html_special_chars($text)
?>