Elm did that. Rust became popular, but people were like “this ain’t English, bruh” so then they copied Elm. Now, finally, tooling for TypeScript can copy Elm.

k, but what’s a specific use case beyond “leveling up TypeScript” you may ask? Great question! For the past few months, I’ve been building tooling around improving cyber security for logs; e.g. ensuring we don’t leak sensitive customer or passwords into our logs.

Most dynamic languages, AND typed languages, will use substring and regex searches in sinks; e.g. where your data leaves your application through a log (console.log, AWS Power Tools, pino, etc). This ensures “the 1 place it exits, we ensure no NPI/PCI leaks occur”. Rad.

… but devs who are into types are like “wait a minute, wait a minute… ‘if it compiles, it works’ if that’s true… then why can’t we ‘if it compiles, it’s safe’?” So they, and me, will leverage types to ensure:

1. sensitive data is typed; could be a Brand, Product, Union… whatever
2. sinks (e.g. logs) do not allow sensitive data

You could do that via types, type proofs, both. So cool, we can now “If it compiles, it is safe”, BUT ALSO “because TypeScript gradually typed and not sound” still have the runtime redaction too.

Except there is one problem: the compiler error messages are trash. Let’s see an example. Below, we’re creating an American Social Security Number (aka ssn) as a Sensitive data brand type. We’re then creating a log function that doesn’t allow it ssn’s to be logged.

So that’s cool; you can see in the image that ssn isn’t compiling when you pass it as a parameter to the log function.

However, what’s NOT cool is the compiler error you get for it: “Type ‘Sensitive‹string, “Social Security Number”›’ is not assignable to type ‘undefined’.”

So never is weird; it’s even weird in Elm. However, it’s used a lot more in TypeScript so you can assert a compiler error. Sometimes, as you can see, the error doesn’t even say never, it says undefined. That said, it still sounds weird when you say that compiler out loud. Usually from a Domain Driven Design perspective you’d be like “A can’t be B” or “a PendingAccount cannot be a ConfirmedAccount”. But… “Cow can’t be assigned to never/undefined” like… wat?

ENTER TYPESCRIPT 7.1 COMPILER API (e.g. import typescript/sync). You can get all the diagnostic errors, and if it’s one you recognize, like “Sensitive SSN to never” you can map it to “Social Security Numbers cannot be logged”

SO BALLER

The 1 cup only had 1 positive, 1 familiar weird, and 1 ultra horrible consequence:

Positive: While a bit tired from lack of sleep, I don’t have caffeine crashes in afternoon.

Weird: dreams getting more intense again; not _as_ intense as no coffee, but I can tell.

Horrible: I wake up, like clockwork, at 4:30 – 5am, every day. I know 6 hours isn’t enough. I’ve tested this on the weekend, and I can nap for up to 2 to 3 hours, so my body is angry or confused about something.

I had depression for about 2 days; was NOT the horribleness that was 8 years ago, but it super in the background, and I didn’t notice it till I really thought about why I was feeling meh. I’m sure sleep deprivation aggravated it, but passed like last weekend.

Despite intense workouts 6 days a week, I wake up early morning, usually from a stomach ache, but I can just tell my brain is WAY more ready to wake up compared to usual if I were to wake up in the middle of the night. Only once in the past 3 weeks of this happening could I fall back asleep till my normal 7am.

I was attempting this to be my first day with no coffee, but I’m going to try another week of just a small cup. I was using those large cups, which I think are like actually “2 cups” (I’ll measure later), and we have this tea cup which is barely 1 cup so… it’s a pretty significant drop. I can tell my body knows what’s coming and is fighting to keep it around. So I’ll give it another week with these little cups, then try again Monday. Hopefully that fixes this weird, new sleep problem. I wasn’t supposed to go backwards, but I have empathy my brain chemistry is just confused from decades of caffeine abuse.

This results in months of building software, but not learning much about building software. This also results in TONS of PR changes requested, and PR’s, even with an AI to help review the PR’s, now go from weeks to months to get merged.

You can see this reflected in this chart from the ArsTechnica article where with AI the number of PR’s goes up, and so too does the number of requested changes. Correlation does not equal causation, no doubt, but I’ve seen it myself, so it resonates and seems a legit finding.

I have a feeling they’re also doing multiple of these large batch, slop grenades at once because they’re waiting on reviews from exhausted Tech Leads / co-workers to review (haven’t checked out their Github profile to conform, tho). The co-workers/Tech Leads are exhausted because:

1. there are more PR’s
2. the PR’s are larger batch
3. the code _looks_ ok at first, but you realize it’s missing fundamental architecture/types/testing good practices, so you have to be on your A game when reviewing.
4. This repeated high-intensity of review tires you out.
5. When done, there are twice as many PR’s like this to go.

Lose-lose for both sides here. Some devs will just tap out, exhausted and “LGTM”. That doesn’t help anybody.

Automation Bias is a thing; people who don’t know a subject well think AI can automate it. In this case Juniors are using a lot of tokens to produce little, usable code.

Microsoft’s Justin Reock, in Developer Experience shared their latest data showing that Juniors use more tokens than Seniors to get the same result. Pre-AI, the data showed the code Seniors wrote tended to last longer whereas code Juniors wrote was changed a lot. That same rule holds now, but the changes have skyrocketed because it’s much easier & faster for them to generate code they don’t understand.

I know many of us feel like the Phoenix from the original X-Men movie when she’s holding back the lakes worth of water WHILE lifting a jet.

You’re trying to help people WHILE not dying yourself. It’s rough. Just know, currently, these AI’s aren’t learning and are making some things worse. However, your efforts on _people_ are worth it; they learn, the slop gets less, and after a couple of months, they can help you. Keep going.

I’m reviewing a mid-level dev’s PR who’s not on our team. First concern is its on our most important repo with high consequence of failure. The work could be piloted on a less risky repo, but she chose this one. She may not be aware of that context. Nowhere in the docs does it state that for the LLM’s to know, more is there a “map between repos” for our team for the LLM to have context. Should we build that? Yes. Do we have that now for the LLM? No. Does the dev know this? No. So I explain all that. She had no idea. She questioned the entire approach. This is the good part of iterative development; failing quickly; reducing the time it takes to learn. This is normal and good.

Second concern is it’s quite large. Not just many files, but many new files, some are quite long. I explain even without Github Stacks, we should be doing small PR’s with few changes and architected for a smaller blast radius. She appreciated the review; apparently getting PR reviews, from a human, are rare in some teams. :: face palm :: She had never heard of Github stacks or even just branching off of a branch; I explained the easier version if she wanted to avoid the github complexity rabbit hole that is gh stacks. Newer LLM’s have some of this in their training data, but unless you specify it, I’ve not seen them recommend it unless the repo has rules.

Third, approaching the work this was new for her. I don’t know what team experiences she had, nor where they are in in their CD journey, but I explained we don’t really plan that out. The whole point is to take small steps, learn, and use those learnings to inform where we should go next. So I don’t really plan or even know what the next PR’s will be, their size, nor the files. That said, I looked at her original PR and gleaned 3 smaller PR’s with only a few changes each that might form a great exercise to learn how to work in small steps since she had already trod this path once AND it’d be easier for me to have a more focus discussion on architecture since the changes are small. The LLM’s can help here if you know how to ask; I don’t think she even know you could ask such a thing yet.

Fourth, there were various architecture challenges, many of which she’d have no context unless she was on our team. Much of the code base was OOP, little dependency injection, heavy use of mocks, little types, and a consistent use of exceptions to drive responses. I’m attempting to move the team towards a more functional, typed, and errors as values approach. LLM’s _really_ struggle with navigating this as each integration has a flurry of options, and unless you narrow each one, it’ll just best guess. This is compounded in Brownfield code bases. This takes 110% of my focus just to help navigate this large space frigate through an asteroid field while drinking my coffee using sketchy inertial dampeners. There’s no way a Junior or Mid-level could both navigate this, know all the options, nor the vision not being on our team. I suggested we pair program on this so I could show how this vision has challenge, and what to do when you’re presented with a crossroads of architecture decisions to lead by example.

Fifth, one architecture one that was prevalent was a lack of DRY around a new framework they are using. It also uses exceptions, but checks the 5 types via instanceof; kind of like their own version of Promise.then/catch. The issue is all the parsing code is on the developer; since they didn’t use error types, you get error:unknown, and about 20+ lines of type narrowing code ensues in at least 4 call sites. At first I’m like “use types” to prevent all this. Then I was like “well, geez… ok, now you need a Zod schema to at least parse to one of the error shapes you’re expecting”. Then I was like “wait a minute, you’re doing this all over the place, why isn’t the framework handling this all for you via abstractions and types?”. Errors as types, types as values, Zod schemas parsing to known shapes, and leaky abstractions are all new concepts to her, and when you throw all of these at once, it’s quite overwhelming. We had a good brief discussion on perhaps making some new PR’s to the framework itself to help here since many others are going to run into the same implementation challenge. The framework has 200% docs, rules, and skills setup for LLM’s to be mostly Spec Driven Development style, but none of that was linked here in our repo, in her rules, or in however this was coded, LLM or not, I suppose. Again, the LLM is having to integrate this framework which has a different style into our repo which has 2 styles, with very little guidance on what to do at crossroads, so you need human in the loop guidance here.

Sixth, the shotgun parsing and lack of an anti-corruption layer was heavy, which in turn caused a lot of shotgun parsing because the maybe’s/optional’s were everywhere. In Brownfield, that’s fine, you just slowly add the parsing higher up, leaving it in a better place than you found it. But in the Greenfield code we’re adding, you have an opportunity to do things better, while still interfacing with the old in clear boundaries. So I explained Parse, Don’t Validate, Shotgun Parsing, and how you “design for the types you want, not what you’re given by some back-end” and how Zod can handle all this in colocated code. Again, it’s a lot throw at someone who’s never heard of all this a cohesive concept. LLM’s, even Opus 3.5, has all this in its training data. It may not be aware of Zod Codecs or the encoding angle, but even ghetto-fabulous SWE 1.6 can pick that up quickly if you give it a golden example. You have to be clear, though, about all of those rules, AND how to integrate them, (e.g. use Zod .transform/.pipe to map from downstream Data Transfer Objects, aka DTO’s ,to our domain Value Objects aka VO’s, so nice types come out).

Seventh, the LLM she was using read our code and started throwing custom class exceptions. The miss was we have a mapping layer at the top to ensure the responses match what we tell the user, so both the exception has to be shaped a certain way, the tests have to check for specific id’s, AND placement has to be thought out because since there are no types, the tests have to be aware of what level this is thrown from to test it. I explained we’re moving away from this approach to Railway programming using Neverthrow to have typed errors. I still can’t tell if this landed, but I know LLM’s can help you a ton here; I still use LLM’s to help debug Promise/ResultAsync chains when the return type isn’t requite right; they have an uncanny ability to fix it, even if you use the wrong explicit return type. Errors as Values is a huge mind shift for those used to imperative coding, especially those not from Go or Rust or FP languages. Especially tricky is the boundaries we have to map back to exceptions so our harness to “read all the errors and translate to the correct user response”. Easy with a .map, hard if you’re like “why do I need a .map?” LLM’s can grok that part easily, what’s hard for them is identifying the correct boundary. I don’t know all the boundaries, nor does she; we’d have to explore together to figure them out.

Eighth, and this is extremely common among TypeScript devs not from an FP background, the types are more for spelling, not for doing any type of Type Driven Development, or “if it compiles, it works” philosophy. So I had to give multiple ideas for certain sections on what could be done to offload to the type system instead of numerous tests, mocks or not. Some situations were impossible, and I had to give a couple of examples to show how you can model it so those impossible situations don’t occur anymore. There were a few switch statements that had default in them, and I had to explain how exhaustiveness checking in TypeScript works, and how the LLM’s will tell you to use never, but ignore them, and how some of that wasn’t her fault because the data they were using was draw DTO’s with optionals and nulls, and some approaches on how to possibly refactor and fix that. The LLM’s are extremely aware of how to type well, and they can easily bounce between OCAML pragmatism eschewing currying, to full Scala (feels like Java) where you don’t need tests “because we typed the universe”. You have to know those styles, AND the verbosity that TypeScript entails, AND figure out where your team falls in that extremely long spectrum, AND what to type now vs later given TypeScript is gradually typed. All of that, and if you’re learning this? Oh man, it’s harder in TypeScript because it’s so verbose compared to other languages, so the LLM (or me) has to know the level you’re at to teach it. Whew!

Had I not reviewed the code:

– the blast radius was high for this PR, with no easy ability to identify what would have caused it given so many changes

– condoning this behavior, human or LLM, sets a precedent that’s ok to make large PR’s. It’s not ok, and I will not condone it.

– would have missed so many wonderful opportunities to teach; I even got a message saying “Thanks for the review, those are rare nowadays” which about made my heart sink; happy she appreciated the attention, but this should be the norm to teach, not to be a meat proxy and just “have the LLM approve it”

– she now knows of all of this to better drive the LLM; before she wouldn’t even think to ask, and the code style + lack of rules-in-repo ensured the LLM would never volunteer alternatives

I got to help teach, she got setup for success, we both learned how to better drive the LLM’s. THIS is why I read the code.

2nd case is a lot simpler. Junior has a small PR to move a service to a new one with a feature flag. This new service is GraphQL, not REST, but the idea is the data will come back and map to our same domain objects. The issue, however, is despite using Zod in the new code, he’s still got Shotgun Parsing going on. If Zod safeParse works, he returns void, and continues on vs. using that safe-to-use domain data. There’s a fundamental understanding of how data validation should work, how TypeScript type narrowing works, and an LLM misunderstanding of how our code base wants you to do things given it’s brownfield and is lacking repo rules.

Again, I get to level him up, and we both learned how we could better articulate to the LLM how to do things, both in prompts and rules, and adding yet another golden data set to give to LLM’s for the future. THAT again, is why I have to read the code.

Yes, LLM’s are helping me. I’ve got Claude and Codex running my 3 Domain Driven Design, Unit Test, and Architecture review skills as well as SWE 1.6 in Windsurf. We have an Enterprise LLM code review that I’ve contributed to as well when I saw a core review was a bit off. So all hands are a deck for this, and helping. None of them, though, know the purpose, or does the Mid-level dev nor Junior know some of these fundamentals in architecture to ASK the LLM’s for help. Together, we all get smarter, and help the LLM’s help us more.

Yes, there are cases where I’m not reading the code; libraries that we use, integration points that never change, infra configurations that a multitudes of red tape and cyber checks to ensure they work and are safe. However, I’m erring on the side of read all of it because I’m consistently seeing code and architecture I wouldn’t want to be on call for in production, there are so many wonderful opportunities to teach and learn myself, misunderstandings from both Dev’s and LLM’s, and all of these help me refine the LLM’s to help me, and hopefully others self-teach, in this regard.

Finally, the most important part, is to learn. Are these requirements correct? Does this architecture allow future change more easily? Is this safe, or will we get dinged by cyber? I could go on and on; so far, the LLM’s aren’t catching all of these things, or are confused on when to apply my rules correct to all situations.

Spending the last 3 weeks side-of-desk looking into redaction, specifically around PCI/NPI (Payment Card Industry / Non Public Information) data (bank account, personal user data, etc) in logs. You break various laws when this happens, and it’s not a fun thing to clean it up.

This in turn taught me that logs aren’t the only sinks, e.g. exits of data out of your app that could have PCI/NPI data in them. Streaming data (e.g. Kinesis, Kafka, etc). HOW you get that data is also interesting. Sometimes you parse stuff through JSON.parse from a downstream, but get things you didn’t mean to. Sometimes a REST call comes in, and things you didn’t intend to keep around were from the headers. Some of this data can go into the sinks that you didn’t intentionally mean to log/stream.

I looked at how most are solving it. The dynamic language, or “lol, wat are types” crews are basically attacking with various Regular Expressions, and string parsing. Any string you log, they either abstract things atop OOP loggers, or write single points of exit sinks like in Pino/Tap. At these points, they’ll combine multiple RegExp’s together into a mass string replacer. I’ve even see a strong arm approach of JSON.stringify’ing an Object, then redacting the big string that comes out.

Some will mark that they redacted something. Some will mark what they redacted. Some will just say “[REDACTED]” and you don’t know if it was a bank account number (BAN), an email, or a number you wrote that was a false positive.

The types crew will use… types. They’ll either create Branded or Variant/Sum/Union types to designate something as a secret, and ensure logging, streaming, and metrics functions cannot take those types. Some will use type proofs in functions or methods to ensure even nested type records do not contain secrets; ensuring “if it compiles, it doesn’t leak PCI data”.

Some, particularly in the TypeScript, Python (gradually typed community) will use both. This is because the type system can, and will often, lie because developers consuming the library can hack their way around it using the 20 billion escape hatches & levers these type systems give you.

The amount of Regular Expression and string matchers is vast. I remember back in the video sharing with chat days, there were many string matchers to filter out bad words from chat, and many also included XSS helpers. This ensures that people who find they can’t type “badword” will type “b a d w o r d” and find that using spaces like that defeats the string matcher for bad words, so they can type other bad words, or even bad code snippets using spaces. The devs will counter with more aggressive string matchers and regular expressions, and they seem to err on the side of “false positives are fine because the cost of failure is quite bad”. While the chat will filter out random words you don’t think are bad, it gives business piece of mind from legal ramifications of NOT doing it. The User Experience costs here are interesting, both to users and to devs.

I’m finding my in own work, for our existing logger, I’m having to use that dual approach of “use types to prevent this from happening”, but also using a combination of regular expressions when the narrowness of type system breaks down. I’m also thinking of starting from scratch on a secondary implementation once the first is done that type narrow specifically to ensure we don’t even need the runtime checks (well, it’s TypeScript, almost anything can be defeated with an as unknown so I’ll have to run various tests). But it’s clear the hybrid approach causes the API to suffer because you’re constraining what the dev can log, but also have to do a ton of type work to compensate for when they (unknowingly) lie to you.

I really like the clean, typed approach as it lessens the attack surface, and I share a lot of this with the streaming and metrics use cases as well as those are less open to the developer so easier to constrain what gets in. It also helps when we attack the problem at the source, and immediately decode into secrets vs. “I don’t know man… they could be anywhere, let’s just JSON.stringify all the things and run a ball of RegExp on it”. Having schema libraries like Zod decode immediately to know secrets enables such a better security posture.

Before: “Where are your secrets?” “Uh, in this codebase… I think from a downstream REST call. You’ll have to go track down their Swagger, it may/may not point them out”

After: “Find all instances of the Secret class.”

I’ve found that some secrets aren’t even secrets. Devs will put credit card number in a secret container, and when you look inside, it’s actually a null. Having a schema validate it’s even an actual secret prevents a lot of this from even being a problem.

There’s also some tricky situations that reminds me of the 1989 Batman movie Joker plot. You’ll have a user’s last name and the last four digits of their credit card. Both aren’t necessarily PCI on their own… but together? Yes. Same with the chemicals Joker was pushing in the movie; isolated they were innocuous, but together, poison. There are various Intersection and Union types you can create to prevent these combinations so you can relax when it compiles.

It’s been interesting to tackle this problem from a Brownfield approach. “How do we prevent PCI data leakage?”

I think I’ve settled on improving the logger we have better runtime redaction, with some compile time preventative measures to help move in the right direction over time. A separate, ideal logger for where we’d like to go. This includes helpers to verify we even have secrets in the first place, and if we do, we use a Parse, Don’t Validate + Anti-Corruption Layer in the same place via a Zod transform.

Brownfield: npm/pip update, you’ll be safer.

Brownfield + migrate: move to this logger, you can offload the problem to the compiler, but log sinks/targets/streaming outputs remain the same.

Greenfield: same

Auditing: With schema parsing, we can confidently know what secrets we have, where, and what combinations we’ve forbidden.

The escape hatches have been interesting too. Getting at your source logger, whether that’s AWS Power Tools, or Pino, or Winston or whatever is often needed to configure it, but if it’s done in more than 1 place, you should get nervous. So we have to provide the ability to do that, but also mark that we’ve done that. I can solve this with a Linter.

For the secrets, you need to provide access to them to do business logic, but sometimes you may have to do that in multiple places unlike log/metric/streaming configurations. I … don’t know how to solve this beyond having an LLM track the Branded type that comes out and doing it’s own audit.

Hard problems, lots of ways to approach it. While I’m having fun, I keep reminding myself why I’m doing this:

1. ensure we’re following the law
2. we won’t have any surprise security work to do
3. make this easy on developers now in Brownfield, and a “pit of success” for those in Greenfield

Still more to learn, but what a rabbit hole, man.