Spending the last 3 weeks side-of-desk looking into redaction, specifically around PCI/NPI (Payment Card Industry / Non Public Information) data (bank account, personal user data, etc) in logs. You break various laws when this happens, and it’s not a fun thing to clean it up.

This in turn taught me that logs aren’t the only sinks, e.g. exits of data out of your app that could have PCI/NPI data in them. Streaming data (e.g. Kinesis, Kafka, etc). HOW you get that data is also interesting. Sometimes you parse stuff through JSON.parse from a downstream, but get things you didn’t mean to. Sometimes a REST call comes in, and things you didn’t intend to keep around were from the headers. Some of this data can go into the sinks that you didn’t intentionally mean to log/stream.

I looked at how most are solving it. The dynamic language, or “lol, wat are types” crews are basically attacking with various Regular Expressions, and string parsing. Any string you log, they either abstract things atop OOP loggers, or write single points of exit sinks like in Pino/Tap. At these points, they’ll combine multiple RegExp’s together into a mass string replacer. I’ve even see a strong arm approach of JSON.stringify’ing an Object, then redacting the big string that comes out.

Some will mark that they redacted something. Some will mark what they redacted. Some will just say “[REDACTED]” and you don’t know if it was a bank account number (BAN), an email, or a number you wrote that was a false positive.

The types crew will use… types. They’ll either create Branded or Variant/Sum/Union types to designate something as a secret, and ensure logging, streaming, and metrics functions cannot take those types. Some will use type proofs in functions or methods to ensure even nested type records do not contain secrets; ensuring “if it compiles, it doesn’t leak PCI data”.

Some, particularly in the TypeScript, Python (gradually typed community) will use both. This is because the type system can, and will often, lie because developers consuming the library can hack their way around it using the 20 billion escape hatches & levers these type systems give you.

The amount of Regular Expression and string matchers is vast. I remember back in the video sharing with chat days, there were many string matchers to filter out bad words from chat, and many also included XSS helpers. This ensures that people who find they can’t type “badword” will type “b a d w o r d” and find that using spaces like that defeats the string matcher for bad words, so they can type other bad words, or even bad code snippets using spaces. The devs will counter with more aggressive string matchers and regular expressions, and they seem to err on the side of “false positives are fine because the cost of failure is quite bad”. While the chat will filter out random words you don’t think are bad, it gives business piece of mind from legal ramifications of NOT doing it. The User Experience costs here are interesting, both to users and to devs.

I’m finding my in own work, for our existing logger, I’m having to use that dual approach of “use types to prevent this from happening”, but also using a combination of regular expressions when the narrowness of type system breaks down. I’m also thinking of starting from scratch on a secondary implementation once the first is done that type narrow specifically to ensure we don’t even need the runtime checks (well, it’s TypeScript, almost anything can be defeated with an as unknown so I’ll have to run various tests). But it’s clear the hybrid approach causes the API to suffer because you’re constraining what the dev can log, but also have to do a ton of type work to compensate for when they (unknowingly) lie to you.

I really like the clean, typed approach as it lessens the attack surface, and I share a lot of this with the streaming and metrics use cases as well as those are less open to the developer so easier to constrain what gets in. It also helps when we attack the problem at the source, and immediately decode into secrets vs. “I don’t know man… they could be anywhere, let’s just JSON.stringify all the things and run a ball of RegExp on it”. Having schema libraries like Zod decode immediately to know secrets enables such a better security posture.

Before: “Where are your secrets?” “Uh, in this codebase… I think from a downstream REST call. You’ll have to go track down their Swagger, it may/may not point them out”

After: “Find all instances of the Secret class.”

I’ve found that some secrets aren’t even secrets. Devs will put credit card number in a secret container, and when you look inside, it’s actually a null. Having a schema validate it’s even an actual secret prevents a lot of this from even being a problem.

There’s also some tricky situations that reminds me of the 1989 Batman movie Joker plot. You’ll have a user’s last name and the last four digits of their credit card. Both aren’t necessarily PCI on their own… but together? Yes. Same with the chemicals Joker was pushing in the movie; isolated they were innocuous, but together, poison. There are various Intersection and Union types you can create to prevent these combinations so you can relax when it compiles.

It’s been interesting to tackle this problem from a Brownfield approach. “How do we prevent PCI data leakage?”

I think I’ve settled on improving the logger we have better runtime redaction, with some compile time preventative measures to help move in the right direction over time. A separate, ideal logger for where we’d like to go. This includes helpers to verify we even have secrets in the first place, and if we do, we use a Parse, Don’t Validate + Anti-Corruption Layer in the same place via a Zod transform.

Brownfield: npm/pip update, you’ll be safer.

Brownfield + migrate: move to this logger, you can offload the problem to the compiler, but log sinks/targets/streaming outputs remain the same.

Greenfield: same

Auditing: With schema parsing, we can confidently know what secrets we have, where, and what combinations we’ve forbidden.

The escape hatches have been interesting too. Getting at your source logger, whether that’s AWS Power Tools, or Pino, or Winston or whatever is often needed to configure it, but if it’s done in more than 1 place, you should get nervous. So we have to provide the ability to do that, but also mark that we’ve done that. I can solve this with a Linter.

For the secrets, you need to provide access to them to do business logic, but sometimes you may have to do that in multiple places unlike log/metric/streaming configurations. I … don’t know how to solve this beyond having an LLM track the Branded type that comes out and doing it’s own audit.

Hard problems, lots of ways to approach it. While I’m having fun, I keep reminding myself why I’m doing this:

1. ensure we’re following the law
2. we won’t have any surprise security work to do
3. make this easy on developers now in Brownfield, and a “pit of success” for those in Greenfield

Still more to learn, but what a rabbit hole, man.

I wanted to cover how you’d do that in Linting because now that LLM’s are showing us how “fuzzy things can still give definitive steps to help, with options”

The cop-out is to be like “build it like Elm… I mean Rust even copied that style so it must be good”.

However, I’ve been reviewing JavaScript Linters lately, and the spectrum is pretty deep, but Flint in particular has a nice Elm-like error in interactive mode.

tl;dr; on Flint, it’s a hybrid JavaScript/TypeScript linter: the core and rules are written in TypeScript to make it easier to allow open source contributions, but plugins can be written in native code, like Biome/Oxlint are doing.

Here’s a screenshot attached of Flint finding a linting failure on a loop using for, citing what file, why the linter failed for this particular bit of code, a suggestion of what to do, and a link to learn more. That’s … amazing.

Now imagine if _every_ domain and infrastructure error you created had even one tenth of this meat embedded inside it, with ability to print for non-prod/local dev logging scenarios? Always inspiring to me to see how people interpret errors, specifically linter errors which can be quite fuzzy/squishy, open to interpretation, but need some kind of actionable step to fix it. I really like Flint’s approach here.

1. wrap native fetch with Neverthrow
2. provide nicer, Elm style return types
3. abstract away decoding/enoding for JSON
4. same, but for GraphQL + company’s rules around that

Learnings the past 3 days:

• Zod v3 to v4 has been a nightmare, so at first I just removed Zod and provided a decoder type interface instead b/c writing a Zod wrapper is like 1 line of code
• … but then I found out in v3.25.0 they name spaced the versions, making it easier for each to co-exist
• … so now I’m wondering if I should undo all my work of “abstracting away a Zod dependency
• but then I found out many are moving to Standard Schema and it’s not a too obtuse type to include, thus enabling me to be nice to ArkType and Effect-ts peoplez
• So now I think I can support Zod v4 internally, include an encoder interface for POST style operations, but still not exclude other schema libraries WHILE reserving the right to use Zod internally for various GraphQL schemas
• … but still requiring me to map those ZodErrors to my own type to not “leak Zod out”.

I really don’t like that last one. I may just not do it, install in a Zod v3 project, and “see what happens” in the unhappy paths.

Once done, I can wrangle the insane grossness that is Open Feature while dual-wielding Discriminated Unions of flame and frost brands.

Initial scare as the grease near the place where the 2 exhaust pipes went in burned off the grease I put in there and it smoked. Then some drops from the front drain plug. I’ve learned to tell the difference between “burning oil” smell, “burning coolant smell” and “omg I don’t know what that is burning” smell. I’m wondering if the crappy hex bolt I used didn’t have the flat head flange, thus not making a good water seal… so headed to Ace Hardware for the 20 billionth time. While scary, at least I can get parts within 15 minutes, cheap vs. online OEM which takes a week or more. Bought another one with a flat head I’ll try replacing later.

She “felt” like she’s running better. While I still can’t tell if the $200 suspension upgrade I did improved anything.

I did replace the bushings for them which was way easier than I thought.

Forks are messy af, tho; I’d say only coolant changes are messier. I could certainly tell the bike “liked” the oil change. Coolant’s hard to tell as it’s cooler today, but I know it was good for her. Cleaned the filter; it wasn’t very dirty but those “lifetime” filters need to be cleaned more often just in case, so hopefully she liked the fresh air from the box. The new rear brake lever I installed works. Friend crashed recently because of no good brakes so I tested her a few times “just in case”. (Bent because of crash at motorcross track)

The tires continue to be loved in the dirt; I can just feel the 400lbs “grab” easier and make me freak out less, which in turn makes me more confident, which in turn makes the bike drive better. They say tires can’t compensate for skill, but I don’t know man…

The turn signals and auxiliary lights are… screwed. I keep using epoxy on top of epoxy and it just never lasts. Duct tape, while visually awful, is functionally awesome.

Sadly, the rear sprocket didn’t fit, and just didn’t have the attention span to attempt to exchange online for whatever the right size is. However, testing her bigger, more highway focused sprocket on the motorcross track, she had torque, so… I’ll live.

Race is Friday. 24 hours, 500+ miles. So nervous my mechanic skills have “forgotten” something, or did something wrong I’ll later regret. Best way to learn is the hard way, no doubt, but… geez. I should have done this in my 20’s.

BTW, if you didn’t see it, here is my testing the new knobby tires on mountain, mud, and motorcross.

This is a series of posts I’m writing about using types as another tool in software development, Continuous Delivery, & keeping LLM’s honest. They’re also a design & refactoring tool, a communication tool, and reduce how many tests you have to write.

There are trade offs to types. More to read, comprehend, & you’re often thinking in 2 modes: types + code. Watch Amanda Laucher & Paul Snively from Strangeloop conference “Types vs. Tests”, you’ll how far you can push the types to the limit of your understanding. As a team, you need to decide if it’s worth getting that understanding for the benefits, or “doing something simpler for now”. No shame deciding to be pragmatic & ship code. Learning types will be iterative. You/your team can gradually increase their skill, bredth of use over time, over many releases. It can also be a line in the sand for a target demographic; go read Dom Syme’s stance against Dependent Types in F#.

Types aren’t as popular as infrastructure, IDE’s, tests, or language features. Community is a part of programming, but a small part. Historically (90’s, 2000’s), languages that had better type systems were not adopted for a variety of reasons. That has been slowly changing since the 2010’s, but super slowly. Wonderful resources online to learn w/tight communities. Despite positive impacts to design + software quality, that does not translate to job opportunities or perceived importance to engineering leadership except for rare language communities.

Types have same issues that other programming quality initiatives have; not the norm. We’ve known for decades about positive impacts of code review, automated tests, Continous Delivery, & past 10 years we now finally have mounting evidence supporting what we knew. Yet companies & clients, either ignore the science, don’t know about it, or don’t know how to balance those standard quality practices w/getting actual work done. You’ll have to navigate politics, project & product management, design & UX, sales & marketing of the techniques & implementation details… WHILE doing you’re engineering job. This is especially hard w/types as they’re niche.

Finally, code is continually created, by your team, others, & library authors. Much of if not typed, or not typed to a strong degree. You’ll have to interface with this code. So learning not only “what’s good” for your domain/your code, but also “how best to safely integrate to this other code” will present additional aspects & challenges.

You’ll be confused why everyone hasn’t adopted sum types; that never goes away, a true mystery of the universe.

I say all of this not to dissuade you, but to prepare you. Not all tradeoffs are the same; some are better than others, and types in my experience are worth it. The “hard to learn” is a great problem because this job attracts nerds & we learn everyday. It can be hard and lonely, but it’s worth it.

Card decks ordering details to follow.