• ensure functions always work by learning Pure Functions
• avoid runtime errors by return Maybes
• avoid runtime errors in deeply nested data using Lenses
• avoid runtime errors by return Results
• creating pure programs from pure functions by Pipeline Programming

Why Care?

Writing code that doesn’t break is much like ensuring your teeth remain healthy. Both have proven ways to prevent problems, yet people still act irresponsibly, either by eating bad foods and not brushing, or in programming being lured by “getting things working quickly with what they know”.

For example, to prevent cavities in your teeth, you should avoid food high in sugar, drink public water supplies with fluoride, and brush your teeth twice a day. At least in the USA, many still have teeth problems. The access problem for dental hygiene can be an economic one. For others, they just aren’t responsible and don’t keep up on teeth health.

For programming, I can help with the access problem. This article will provide proven ways to equip you with the knowledge to help yourself.

For the responsibility, however, I empathize it’s harder. I’ll teach you the techniques, but you have to practice them. Brushing your teeth everyday, you can master it quite quickly. Utilizing functional programming techniques in a non-functional programming language takes a lot more practice and hard work.

Your teeth thank you for brushing often. Your code will thank you by consistently working.

What Does “Never Break” Mean Exactly?

The “Never Break” means the function will always work, will always return a value, won’t ever raise an Exception, nor will it ever exit the Python process unless you want it to. It doesn’t matter what order you call these functions or program in, how many times you call it, they are always predictable, and can be depended on.

What Does “Mostly Work” Mean Exactly?

Just because your functions work all the time doesn’t mean your software works. Pass the wrong environment variable for a Docker container, a downstream API is down, or perhaps you just did the math wrong. You still need unit, fuzz, feature tests, formal methods if you’re able, and good ole manual testing to verify your software works. Not having to worry about your software exploding randomly allows you to focus 100% on those.

Also, no amount of FP will save you from badly formatted code. You’ll still need things like PyLint to ensure you wrote print("Sup") instead of print("Sup)

If this is so obvious, when do I _not_ do this?

In order of priority:

1. When you’re exploring ideas.
2. When you’re committing code amongst those who aren’t trained in FP.
3. When you’re on a deadline.
4. When you’re modifying legacy code with no unit tests.

The allure and draw of Python is it is a dynamic language. While not as forgiving as JavaScript, Lua, or Ruby, it still allows a lot of freedom to play with ideas, various data types, and provide a variety of ways to run your code efficiently on various infrastructure architectures. With types only enforced (mostly) at runtime, you can try various ideas, quickly run them, correct mistakes you find after running them, and repeat this until you’ve locked down an implementation. While you _can_ use FP concepts for this, if you’re still learning, they can slow you down. Other times, this is a fun time to learn.

Commiting FP code to a Github repo where others aren’t familiar with FP, or have no clue why you’re coding things that don’t seem PEP Compliant can really cause problems. Typically a team adopts their own rules, patterns, and styles… and they don’t always have reasonable reasons. It’s best to learn why they code the way they do things, and adopt those standards. If you’re in a position to teach the team, great, but FP is quite alien, already has a reputation for being obtuse with horrible evangelists. Tread slowly here. Breaking trust in the team is one way to ensure your software never works correctly. Bad working relationships result in horrible software.

If you’re on a deadline, learning anything new can slow you down and risk not hitting it. Alternatively, it’s also the guaranteed way to ensure you learn something quickly, heh.

FP or not, you shouldn’t add or modify code in a large codebase you’re responsible for if it doesn’t have unit tests. Otherwise, you have no good idea if you’ve broken something, sometimes for days or weeks. Add tests first, THEN refactor things.

Functions That Always Work: Writing Pure Functions

Pure functions always work. They reason they always work is that they always return values. They don’t raise Exceptions. They’re technically not supposed to have side effects. They’re desired because they always return the same value, so are close to math in that you can depend on their result, or “answer”. Unit testing them also doesn’t require mocks, only stubs.

The 2 official rules go like this:

1. same input, same output
2. no side effects

Returning None is ok. The first function most Python devs are introduced too, print doesn’t appear to return value, much like console.log in JavaScript. However, it does: None:

result = print("Sup")
print(result == None) # True

Typically a function that returns no value, or None in Python’s case, is considered a “no operation” function, or “noop” for short (pronounced no-op). Noop’s are usually a sign the function has side effects. Noops are not pure functions. We know that print does produce side effects; the whole point of calling it is to produce the side effect of writing to standard out so we can see what the heck our code is doing.

For classes, however, it’s more subtle and now that you know the rules, you’ll see what’s wrong. Here’s how you stop verifying SSL certs for rest calls using a class to wrap urllib:

import request.rest
import ssl

req = request.rest()
req.disable_ssl()
res = req.get("https://some.server.com")

Note the disable_ssl() class method. It takes no parameters, and returns no value. Why? Probably because like most classes, it’s changing a setting internally in the class instance to turn off SSL stuff so the next person who does a REST call won’t have to have certs validated.

You do the complete opposite in functional programming. Although, in this case, it’s probably ok to call disable_ssl() multiple times without any harm. Things like get are more tricky.

So, impure function:

ssl = enabled

def get(url):
  return requests.get(url, ssl_enabled=ssl)

And a pure function:

def get(ssl, url):
  return requests.get(url, ssl_enabled=ssl)

And one that’s even more pure, and unit testable:

def get(requests, ssl, url):
  return requests.get(url, ssl_enabled=ssl)

And the most pure function you can possibly write in Python in a reasonable way:

def get(requests, ssl, url):
  try:
    result = requests.get(url, ssl_enabled=ssl)
    return result
  except Exception as e:
    return e

Write functions like that, and you’re well on your way to understanding how Golang is written.

Avoiding None by using Maybes

Python doesn’t offer a lot of guarantee’s, that’s why risk takers like it. If you’re writing software, you might not want risk. There are 3 main places this comes from when calling functions:

1. getting data from Dictionaries (cause you’re using them now, not Classes, right?)
2. getting data from Lists
3. getting data from outside of Python

Safer Dictionaries: Maybe Tuple

Let’s talk about dictionaries.

Dictionaries can work:

person = { firstName: "Jesse" }
print(person["firstName"]) # Jesse

Dictionaries can also fail:

print(person["lastName"])
# KeyError: 'lastName'

Wat do!?

You need to change how you think Python in 2 ways. First, how you access objects safely, such as using key in dictionary or lenses. Second, how you return values from functions, such as using the Golang syntax by returning multiple values which lets you know if the function worked or not, or a Maybe/Result type.

You can safely access dictionaries by creating a getter function:

def get_last_name(object):
  if "lastName" in object:
    return (True, object["lastName"], None)
  return (False, None, f"lastName does not exist in {object}")

This function is pure, and safe and will work with any data without blowing up. It also uses a nice trick Python has when you return a Tuple (read only List) from a function; you can destructure it to get 3 variables out in a terse syntax, making it feel like it is returning multiple values. We’ve chosen something similar to the Golang syntax where they return value, error, we’re returning didItWork, value, error. You can use the Golang syntax if you wish, I just don’t like writing if error != None.

ok, lastName, error = get_last_name(person)
if ok == False:
  return (False, None, f"Failed to get lastName from {person}")

So this is your first Maybe at a raw level. It’s a Tuple that contains if the function had your data or not, the data if it did, and if not why. Note if ok is False, your program is probably done at this point.

Developers are encouraged to create Exceptions for everything that isn’t necessarily exceptional, and raise them so others can catch them or different types of them and react accordingly, usually higher up the function chain. The problem with this is you can’t easily read your code and see errors as they could be coming from completely different files. Using maybes in this way, it’s very clear what function failed, and your don’t have to wrap things in try/catch “just in case”.

Safer Dictionaries: Maybe Type

Tuples are ok, but are verbose. A shorter option is the Maybe type. We’ll use PyMonad’s version because they did a lot of hard work for us. First import it:

from pymonad.Maybe import *

Then, we’ll create our getLastName function to return a Maybe type instead of a Tuple like we did before:

def get_last_name(object):
  if "lastName" in object:
    return Just(object["lastName"])
  return Nothing

I say the word “type”, but in Python, it feels like a function. Replace (True, data, None) with Just(data) and (False, None, Exception('reason')) with Nothing. You can then use it:

lastNameMaybe = get_last_name(person)

You first instinct will be “cool, if it’s a Just, how do I get my data out?”. Well, you don’t.

“Wat!?”

Trust me, we’ll cover this in Pipeline Programming below, for now, just know this function will never fail, and you’ll always get a Maybe back, ensuring your code doesn’t throw errors, and is more predictable and testable.

Speaking of which, here’s Pytest:

def test_get_last_name_happy():
  result = get_last_name({'lastName': 'cow'})
  assert result == Just('cow')

“Wat…”

Ok, till you get more comfortable, try this:

def test_get_last_name_happy():
  result = get_last_name({'lastName': 'cow'})
  assert result.value == 'cow'

Safer Lists: Maybe Type

The same thing can be said for Listsin Python.

people = [{'firstName': 'Jesse'}]
first = people[0]

Cool.

people = []
first = people[0]
# IndexError: list index out of range

Uncool. There are many ways to do this; here’s the quickfix you’ll end up repeating a lot:

def get_first_person(list):
  try:
    result = list[0]
    return Ok(result)
  except Exception:
    return Nothing

You’ll see this implemented in a more re-usable way as nth, except instead of returning None, it’ll return a Maybe:

def nth(list, index);
  try:
    result = list[index]
    return OK(result)
  except Exception:
    return Nothing

Pure Deeply Nested Data Using Lenses

You know how to make unbreakable functions by making them pure. You know how to access Dictionaries and Lists safely using Maybes.

In real-world applications, you usually get larger data structures that are nested. How does that work? Here’s some example data, 2 people with address info:

people = [
  { 'firstName': 'Jesse', 'address': { skreet: '007 Cow Lane' } },
  { 'firstName': 'Bruce', 'address': { skreet: 'Klaatu Barada Dr' } }
]

Let’s get the 2nd person’s address safely using a Maybe:

def get_second_street(list):
  second_person_maybe = nth(list, 1)
  if isinstance(second_person_maybe, Just):
    address_maybe = get_address( second_person_maybe.value)
    if isinstance(address_maybe, Just):
      street_maybe = get_street(address_maybe.value)
      return street_maybe
  return Nothing

Yeahh…… no. Gross. Many have taken the time to do this with a better, easier to use API. PyDash has a get method:

from pydash import get

def get_second_street(list):
  return get(list, '[1].address.skreet')

Cool, eh? Works for Dictionaries, Lists, and both merged together.

Except… one small issue. If it doesn’t find anything, it returns a None. None will cause runtime Exceptions. You can provide a default as the 3rd parameter. We’ll wrap it with a Maybe; less good looking, but MOAR STRONG AND PURE.

def get_second_street(list):
  result = get(list, '[1].address.skreet')
  if result is None:
    return Nothing
  return Just(result)

Returning Errors Instead of Raising Exceptions

Dictionaries and Arrays not having data is ok, but sometimes things really do break or don’t work… what do we do without Exceptions? We return a Result. You have 2 options on how you do this. You can use the Tuplewe showed you above, doing it Golang style:

def ping():
  try:
    result = requests.get('https://google.com')
    if result.status_code == 200:
      return (True, "pong", None)
    return (False, None, Exception(f"Ping failed, status code: {result.status_code}")
  except Exception as e:
    return (False, None, e)

However, there are advantages to use a true type which we’ll show later in Pipeline Programming. PyMonad has a common one called an Either, but Left and Right make no sense, so I made my own called Result based on JavaScript Folktale’s Result because “Ok” and “Error” are words people understand, and associate with functions working or breaking. Left and Right are like… driving… or your arms… or dabbing… or gaming… or anything other than programming.

def ping():
  try:
    result = requests.get('https://google.com')
    if result.status_code == 200:
      return Ok("pong")
    return Error(Exception(f"Ping failed, status code: {result.status_code}"))
  except Exception as e:
    return Error(e)

You don’t have to put Exceptions in Error; you can just put a String. I like Exception’s because they have helpful methods, info, stack trace info, etc.

Pipeline Programming: Building Unbreakable Programs

You know how to build Pure Functions that don’t break. You can safely get data that has no guarantee it’ll be there using Maybes and Lenses. You can call functions that do side effects like HTTP requests, reading files, or parsing user input strings safely by returning Results. You have all the core tools of Functional Programming.. how do you build Functional Software?

By composing functions together. There are a variety of ways to do this purely. Pure functions don’t break. You build larger functions that are pure that use those pure functions. You keep doing this until your software emerges.

Wait… What do you mean by “Composing”?

If you’re from an Object Oriented Background, you may think of Composing as the opposite of Inheritance; uses class instances inside of another class. That’s not what we mean here.

Let’s parse some JSON! The goal is to format the names of humans from a big ole List of Dictionaries. In doing this you’ll learn how to compose functions. Although these aren’t pure, the concept is the same.

Behold, our JSON string:

peopleString = """[
	{
		"firstName": "jesse",
		"lastName": "warden",
		"type": "Human"
	},
	{
		"firstName": "albus",
		"lastName": "dumbledog",
		"type": "Dog"
	},
	{
		"firstName": "brandy",
		"lastName": "fortune",
		"type": "Human"
	}
]"""

First we must parse the JSON:

def parse_people(json_string):
  return json.loads(json_string)

Next up, we need to filter only the Humans in the List, no Dogs.

def filter_human(animal):
  return animal['type'] == 'Human'

And since we have a List, we’ll use that predicate in the filter function from PyDash:

def filter_humans(animals):
  return filter_(animals, filter_human)

Next up, we have to extract the names:

def format_name(person):
  return f'{person['firstName']} {person['lastName']}'

And then do that on all items in the List; we’ll use map from PyDash for that:

def format_names(people):
  return map_(people, format_name)

Lastly, we need to upper case all the names, so we’ll map yet again and start_case from PyDash:

def uppercase_names(people):
  return map_(people, start_case)

Great, a bunch of functions, how do you use ’em together?

Nesting

Nesting is the most common.

def parse_people_names(str):
  return uppercase_names(
    format_names(
      filter_humans(
        parse_people(str)
      )
    )
  )

Oy… that’s why you often hear “birds nest” being the negative connotation to describe code.

Flow

While PyDash and Lodash call it flow, this is the more common way to build larger functions out of smaller ones via composing them, and gives you your first insight into “pipeline” style programming.

parse_people = flow(parse_people, filter_humans, format_names, uppercase_names)

Pipeline: PyMonad Version

Now Flow is quite nice, but hopefully you saw some problems. Specifically, none of those functions are super pure. Yes, same input, same output and no side effects… but what happens when one of them returns a Nothing? When happens if you’re doing dangerous stuff and one returns a Result that contains an Error instead of an Ok?

Well, each of those types are tailor made to pipe together. You saw how the functions I made for flow worked together; they just needed 3 rules:

1. be a mostly pure function
2. have a single input
3. return an output

The Maybe and Result can wired together too, but they have a few extra special features. The only 4 we care about for this article are:

1. if a Maybe gets a Just, it’s smart enough to get the Just(thing).value and pass it to the next function. The Result is the same unwrapping the value in the Ok and and passing it to the next function.
2. Each expects you to return the same type back. If you chain Maybe‘s together like you do in flow, then it’s expected you return your Just(thing) or Nothing.
3. Both handle bad things. If a chain of Maybe‘s suddenly gets a Nothing, the entire thing will give you a Nothing. If any of the functions you’ve wired together get a Result and suddenly one gets an Error in the chain, the entire chain gets an Error.
4. They have flow built in; but instead of calling it, you use weird, new, non-Python symbols to confuse you, look impressive, and make the code feel less verbose despite increased brain activity.

That’s a lot, ignore it. Just look at the example:

def parse_people_names(str):
  return parse_people(str) \
  >> filter_humans \
  >> format_names \
  >> uppercase_names

Goodbye!

If that’s a bit alien and strange, that’s because:

• you’re doing Functional Programming in a Object Oriented + Imperative language; you rock!
• >> isn’t Pythonic nor PEP Compliant®
• Most Python devs see a \ and think “Aw man, this code is too long…”
• “… wait, you’re putting functions there, but not calling them, nor giving them parameters, what in the…”

This is why many Functional Programmers are nice even if they are not-so-great evangelists. Many non-FP destined people see that kind of code, freakout and leave. This makes many FP’ers lonely, thus they are more than happy to be cordial and polite when people talk to them in the programming community.

Manual Pipeline

Remember that nasty example of getting deeply nested properties before you learned Lenses? Let’s replace that with a pipeline using a Maybe; it’ll give you a better sense of how these things are wired together, like flow is above.

def get_second_street(list):
  second_person_maybe = nth(list, 1)
  if isinstance(second_person_maybe, Just):
    address_maybe = get_address( second_person_maybe.value)
    if isinstance(address_maybe, Just):
      street_maybe = get_street(address_maybe.value)
      return street_maybe
  return Nothing

Gross. Ok, let’s start from the top, first we need lastName (btw, I’m glad you ignored the ‘Goodbye’ and are still here, YOU GOT WHAT IT TAKES, OH YEAH):

def get_second_person(object):
  return nth(object, 1)

Cool, next up, get the address:

def get_address(object):
  if 'address' in object:
    return Just(object['address'])
  return Nothing

Finally, get dat skreet skreet!

def get_street(object):
  if 'skreet' in object:
    return Just(object['skreet']
  return Nothing

Now let’s compose ’em together.

def get_second_street(object):
  second_person_maybe = get_second_person(object)
  if isinstance(second_person_maybe, Nothing):
    return Nothing

  address_maybe = get_address(second_person_maybe.value)
  if isinstance(address_maybe, Nothing):
    return Nothing

  street_maybe = get_street(address_maybe)
  if isinstance(street_maybe, Nothing)
    return Nothing

  return street_maybe

Essh… ok, let’s test her out:

second_street_maybe = get_second_street(people)

Note a couple things. Each time you call a function, you then inspect if the return value is a Nothing. If it is, you immediately just return that and stop running everything else. Otherwise, you call the next one in the chain and unwrap the value. Here we’re doing that manually via maybe.value. Also, the return street_maybe at the end is a bit redundant; no need to check for Nothing there, just return it, but I wanted you to see the repeating pattern 3 times.

That pattern is what the >> does for you: checks for Nothing and aborts early, else unwraps the value and gives to the function in the chain. Rewriting it using that bind operator:

def get_second_street(object):
  return get_second_person(object) \
  >> second_person_maybe \
  >> get_address \
  >> get_street

It’s easy to forget the \. This is because Python is super strict on whitespace and only occasionally lets you do line breaks with code.

If you don’t want to use that, just put her all on 1 line:

def get_second_street(object):
  return get_second_person(object) >> second_person_maybe >> get_address >> get_street

Manual Result Pipeline

Let’s do some dangerous shit. We’re going to load our encryption key from AWS, then fetch our OAuth token, then load in a list of Loan products from an API that requires that token to work, and finally snag off just the ID’s.

Dangerous stuff? Potential Exceptions? A job for Result.

Get dat key:

def get_kms_secret():
  try:
    result = client.generate_data_key(
      KeyId='dev/cow/example'
    )
    key = base64.b64decode(result['Plaintext']).decode('ascii')
    return Ok(key)
  except Exception as e:
    return Error(e)

Don’t worry if you don’t know what the heck KMS is, it’s just Amazon’s encryption stuff, and gives you keys that you’re only allowed to get. If you’re not, that function will fail. It just gives us a temporary private encryption key as text. We’ll use that to get our OAuth token.

Next up, get our OAuth token via the requests library:

def get_oauth_token(key):
  try:
    response = requests.post(oauth_url, json={'key': key})
    if response['status_code'] == 200:
      try:
        token_data = response.json()
        return Ok(token_data['oauth_token'])
      except Exception as parse_error:
        return Error(parse_error)
    return Error(Exception(response.text))
  except Exception as e:
    return Error(e)

You have a style choice here. If you look above, there are 4 potential failures: status code not being a 200, failing to parse the JSON, failing to find the oauth token in the JSON you parsed, or a networking error. You can just “handle it all in the same function” like I did above. The point of mashing it together is “Did I get a token or not?”. However, if you don’t like the nested ifs and trys then you can split that into 4 functions, each taking a Result as well to wire them together in order.

Now that we have our token, let’s call the last API to get a list of Loan products. We’ll get a list potentially, but all we want is the id’s, so we’ll map to pluck those off:

def get_loan_ids(token):
  try:
    auth_header = {'authentication': f'Bearer {token}'}
    response = requests.get(loan_url, headers=auth_header)
    if response.status_code == 200:
      try:
        loans = response.json()
        ids = map_(loans, lambda loan: get(loan, 'id', '???'))
        return Ok(ids)
      except Exception as e:
        return Error(e)
    return Error(Exception(response.text))
  exception Exception as overall_error:
    return Error(overall_error)

If everything goes well, you’ll get a list of strings. Otherwise, and Error. Let’s wire all 3 up:

def get_loan_ids():
  return get_kms_secret() \
  >> get_oauth_token \
  >> get_loan_ids

When you go:

loan_ids_result = get_load_ids()

Either it works, and that loan_ids_result is an Ok, or it failed somewhere in there and it’s an Error containing the Exception or error text.

… now, one cheat I did, and you’re welcome to mix and match Maybes and Results together. You see when we attempt to get the loan id?

get(loan, 'id', '???')

That 3rd parameter is a default value if the property isn’t there or is None. The _right_ thing to do is use a Maybe instead. You can be pragmatic like this if you wish, just be aware, these are the types of things you’ll see where “the code has no errors but doesn’t work” . Also, I tend to like Errors more than Nothing in these scenarios because they give you an opportunity to provide an Exception or text with a lot of context as to WHY. Why is huge for a programmer, especially when you’re close to the error and know why it probably failed in a large set of functions that could all possibly break.

Conclusions

Functions are fine, but Exceptions can cause a lot of indirection. Meaning, you think you know the 3 ways a function, or even a program can fail, but then another unexpected Exception comes from some other deeply nested function. Completely removing these, and locking down the ones you know, or don’t know, will blow up using try/catch via pure functions removes that problem for you. You now know for a fact what paths your functions take. This is the power of pure functions.

However, that doesn’t protect you from using null (i.e. None) data. You’ll get all kinds of runtime Exceptions in functions that were expecting good data. If you force all of your functions to handle that eventuality via Maybes, then you never get null pointers. This includes using Lenses to access deeply nested data.

Once you go outside of your program via side effects such as making REST calls, reading files, or parsing data, things can go wrong. Errors are fine, and good, and educational. Having a function return a Result instead of raising an Exception is how you ensure the function is pure. You can also abort early if there is a problem vs having cascading effects with other functions not having their data in a state they need. You also, much like Golang and Rust, have the opportunity to document what the error is where it happens with helpful context vs. guessing at a long stack trace.

Finally, once you’ve written pure functions, you can build larger pure functions that compose those together. This is how you build pure software via pipeline programming (also called railroad programming, a style of streaming, etc).

Python provides the PyDash library to give you the basics of pure functions with working with data and lists. PyMonad provides the basics in creating Maybe and Either (what I call Result). If you’re interested, there is the Coconut programming language that integrates and compiles to Python, allowing you to write in a more Functional Programming style. Here’s a taste re-writing our example above:

def get_loan_ids():
  return get_kms_secret()
  |> get_oauth_token
  |> get_loans
  |> map( .id )

Don’t fret if all of this seems overwhelming. Functional Programming is a completely different way of thinking, Python is not a functional language, and these are all super advanced concepts I just covered the basics of. Just practice the pure functions and writing tests for them. Once you get the basics of that, you’ll start to get a feel, like a Spidey Sense, of when something is impure and has side effects. With Maybes, you’ll love them at first, then realize once you start using them responsibly, how much extra work you have to do to avoid null pointers. Eithers/Results can be challenging to debug at first as you won’t be wading through stacktraces, and you learn what error messages are best to write, and how to capture various failures. Stick with it, it’s ok if you use only a little; the little you use will still help your code be more solid.

There are various ways of handling errors in programming. This includes not handling it. Many languages have created more modern ways of error handling. An error is when the program, intentionally, but mostly not, breaks. Below, I’ll cover the 4 main ones I know: try/catch, explicit returns, either, and supervising crashes. We’ll compare various languages on how they approach errors: Python, JavaScript, Lua, Go, Scala, Akka, and Elixir. Once you understand how the newer ways work, hopefully this will encourage you to abandon using potentially program crashing errors via the dated throw/raise in your programs.

Contents

• Why Even Care
• Throw, Try, Catch, Finally
• Explicit Return
  • Various Language Examples of Explicit Returns
  • Effects On Coding
  • Cons on Explicit Returns
• Either, Left or Right, and Pattern Matching
  • The Either Type
  • Either Examples
  • Pattern Matching
• Let It Crash
• Conclusions

Why Even Care?

When programs crash, log/print/trace statements and errors aren’t always helpful to quickly know what went wrong. Logs, if there are at all, tell a story you have to decipher. Errors, if there, sometimes give you a long stack trace, which often isn’t long enough. Asynchronous code can make this harder. Worse, both are often completely unrelated to the root cause, or lie and point you in the completely wrong debugging direction. Overall, most crashes aren’t always helpful in debugging why your code broke.

Various error handling strategies can prevent these problems.

Throw, Try, Catch, Finally

The original way to implement an error handling strategy is to throw your own errors.

// a type example
validNumber = n => _.isNumber(n) && _.isNaN(n) === false;
add = (a, b) => {
   if(validNumber(a) === false) {
     throw new Error(`a is an invalid number, you sent: ${a}`);
   }
   if(validNumber(b) === false) {
     throw new Error(`b is an invalid number, you sent: ${b}`);
   }
   return a + b;
};
add('cow', new Date()); // throws

They have helpful and negative effects that take pages of text to explain. The reason you shouldn’t use them is they can crash your program. While often this is often intentional by the developer, you could negatively affect things outside your code base like a user’s data, logs, and this often trickles down to user experience. It also makes it more difficult for the developer debugging to pinpoint exactly where it failed and why.

I jokingly call them explosions because they accidentally can affect completely unrelated parts of the code when they go off. Compilers and runtime errors in sync and async code still haven’t gotten good enough (except maybe Elm) to help us immediately diagnose what we, or someone else, did wrong.

We don’t want to crash a piece of code or entire programs.

We want to correctly identify what went wrong, give enough information for the developer to debug and/or react, and ensure that error is testable. Intentional developer throws attempt to tell you what went wrong and where, but they don’t play nice together, often mask the real issue in cascading failures, and while sometimes testable in isolation, they are harder to test in larger composed programs.

Explicit Return

The second option is what Go, Lua, and sometimes Elixir do, where you handle the possible error on a per function basis. They return information if the function worked or not along with the regular return value. Basically they return 2 values instead of 1. These are different for asynchronous calls per language so let’s focus on synchronous for now.

Various Language Examples of Explicit Returns

Lua functions will throw errors just like Python and JavaScript. However, using a function called protected call, pcall it will capture the exception as part of a 2nd return value:

function datBoom()
  error({reason='kapow'})
end
ok, error = pcall(datBoom)
print("did it work?", ok, "error reason:", error.reason)
-- did it work? false, error: kapow

Go has this functionality natively built in:

func datBoom() (err error)
ok, err := datBoom()
if err != nil {
  log.Fatal(err)
}

… and so does Elixir (with the ability to opt out using a ! at the end of a function invocation):

def datBoom do
  {:error, "kapow"}
end

{:error, reason} = datBoom()
IO.puts "Error: #{reason}" ## kapow

While Python and JavaScript do not have these capabilities built into the language, you can easily add them.

Python can do the same using tuples:

def datBoom():
  return (False, 'kapow')

ok, error = datBoom()
print("ok:", ok, "error:", error) # ('ok:', False, 'error:', 'kapow')

JavaScript can do the same using Object destructuring:

const datBoom = () => ({ok: false, error: 'kapow'});
const {ok, error} = datBoom();
console.log("ok:", ok, "error:", error); // ok: false error: kapow

Effects On Coding

This causes a couple of interesting things to happen. First, developers are forced to handle errors when and where they occur. In the throw scenario, you run a lot of code, and sprinkle throws where you think it’ll break. Here, even if the functions aren’t pure, every single one could possibly fail. There is no point continuing to the next line of code because you already failed at the point of running the function and seeing it failed (ok is false) an error was returned telling you why. You start to really think how to architect things differently.

Second, you know WHERE the error occurred (mostly). The “why” is still always up for debate.

Third, and most important, if the functions are pure, they become easier to unit test. Instead of “I get my data, else it possibly blows up”, it immediately tells you: “I worked, and here’s your data”, or “I broke, and here’s what could be why”.

Fourth, these errors DO NOT (in most cases if your functions are pure) negatively affect the rest of the application. Instead of a throw which could take other functions down with it, you’re not throwing, you’re simply returning a different value from a function call. The function “worked” and reported its “results”. You’re not crashing applications just because a function didn’t work.

Cons on Explicit Returns

Excluding language specifics (i.e. Go panics, JavaScript’s async/await), you have to look in 2 to 3 places to see what went wrong. It’s one of the arguments against Node callbacks. People say not to use throw for control flow, yet all of you done is create a dependable ok variable. A positive step for sure, but still not a hugely helpful leap. Errors, if detected to be there, mold your code’s flow.

For example, let’s attempt to parse some JSON in JavaScript. You’ll see the absence of a try/catch replaced with an if(ok === false):

const parseJSON = string => {
  try {
    const data = JSON.parse(string);
    return {ok: true, data};
  } catch(error) {
    return {ok: false, error};
  }
};
const {ok, error, data} = parseJSON(new Date());
if(ok === false) {
  console.log("failed:", error);
} else {
  console.log("worked:", data);
}

Either, Left or Right, and Pattern Matching

The Either Type

Functions that can return 2 types of values are solved in functional programming by using the Either type, aka a disjoint union. Typescript (strongly typed language & compiler for JavaScript) supports a psuedo Either as an Algebraic Data Type (aka ADT).

For example, this TypeScript getPerson function will return Error or Person and your compiler helps you with that:

// Notice TypeScript allows you to say 2 possible return values
function getPerson() : Error | Person

The getPerson will return either Error, or Person, but never both.

However, we’ll assume, regardless of language, you’re concerned with runtime, not compile time. You could be an API developer dealing with JSON from some unknown source, or a front end engineer dealing with user input. In functional programming they have the concept of a “left or right” in an Either type, or an object depending on your language of choice.

The convention is “Right is Correct” and “Left is Incorrect” (Right is right, Left is wrong).

Many languages already support this in one form or another:

JavaScript through Promises as values: .then is right, .catch is left) and Python via deferred values via the Twisted networking engine: addCallback is right, addErrback is left.

Either Examples

You can do this using a class or object in Python and JavaScript. We’ve already shown you the Object version above using {ok: true, data} for the right, and {ok: false, error} for the left.

Here’s a JavaScript Object Oriented example:

class Either {
  constructor(right=undefined, left=undefined) {
    this._right = right;
    this._left = left;
  }
  isLeft() {
    return this.left !== undefined;
  }
  isRight() {
    return this.right !== undefined;
  }
  get left() {
    return this._left;
  }
  get right() {
    return this._right;
  }
}

const datBoom = () => new Either(undefined, new Error('kapow'));
const result = datBoom();
if(result.isLeft()) {
  console.log("error:", result.left);
} else {
  console.log("data:", result.right);
}

… but you can probably already see how a Promise is a much better data type (despite it implying async). It’s an immutable value, and the methods then and catch are already natively there for you. Also, no matter how many then‘s or “rights”, 1 left can mess up the whole bunch, and it allllll flows down the single catch function for you. This is where composing Eithers (Promises in this case) is so powerful and helpful.

const datBoom = () => Promise.reject('kapow');
const result = datBoom();
result
.then(data => console.log("data:", data))
.catch(error => console.log("error:", error));

Pattern Matching

Whether synchronous or not, though, there’s a more powerful way to match the Either ‘esque types through pattern matching. If you’re an OOP developer, think of replacing your:

if ( thingA instanceof ClassA ) {

with:

ClassA: ()=> "it's ClassA",
ClassB: ()=> "it's ClassB".

It’s like a switch and case for types.

Elixir does it with almost all of their functions (the _ being the traditional default keyword):

case datBoom do
  {:ok, data}      -> IO.puts "Success: #{data}"
  {:error, reason} -> IO.puts "Error: #{reason}"
  _                -> IO.puts "No clue, brah..."
end

In JavaScript, you can use the Folktale library.

const datBoom = () => Result.Error('kapow');
const result = datBoom();
const weGood = result.matchWith({
  Error: ({value}) => "negative...",
  Ok: ({value}) => "OH YEAH!"
});
console.log("weGood:", weGood); // negative...

Python has pattern matching with Hask (although it’s dead project, Coconut is an alternative):

def datBoom():
  return Left('kapow')

def weGood(value):
    return ~(caseof(value)
                | m(Left(m.n)) >> "negative..."
                | m(Right(m.n)) >> "OH YEAH!")

result = datBoom()
print("weGood:", weGood(result)) # negative...

Scala does it as well, looking more like a traditional switch statement:

def weGood(value: Either): String = value match {
  case Left => "negative..."
  case Right => "OH YEAH!"
  case _ => "no clue, brah..."
}
weGood(Left('kapow'))  // negative...

Let It Crash

The Mathematicians came up with Either. Three cool kats at Ericsson in 1986 came up with a different strategy in Erlang: let it crash. Later in 2009, Akka took the same idea for the Java Virtual Machine in Scala and Java.

This flies in the face of the overall narrative of this article: don’t intentionally cause crashes. Technically it’s a supervised crash. The Erlang / Akka developers know errors are a part of life, so embrace they will will happen, and give you a safe environment to react to them happening without bringing down the rest of your application.

It also only becomes relatable if you do the kind of work where uptime with lots of traffic is the number one goal. Erlang (or Elixir) create processes to manage your code. If you know Redux or Elm, the concept of a store to keep your (mostly) immutable data, then you’ll understand the concept of a Process in Elixir, and an Actor in Akka. You create a process, and it runs your code.

Except, the framework developers knew that if you find a bug, you’ll fix it and upload new code to the server. If the server needs to keep running to serve a lot of customers, then it needs to immediately restart if something crashes. If you upload new code, it needs to restart your new code as the older code processes shut down when they are done (or crash).

So, they created supervisors Elixir|Scala. Instead of creating 1 process that runs your code, it creates 2: one to run your code, and another to supervise it if it crashes, to restart a new one. These processes are uber lightweight (0.5kb of memory in Elixir, 0.3kb in Akka).

While Elixir has support for try, catch, and raise, error handling in Erlang/Elixir is a code smell. Let it crash, the supervisor will restart the process, you can debug the code, upload new code to a running server, and the processes spawned from that point forward will use your new code. This is similar to the immutable infrastructure movement around Docker in Amazon’s EC2 Container Service and Kubernetes.

Conclusions

Intentionally crashing programs is a bad programming practice. Using throw is not the most effective way to isolate program problems, they aren’t easy to test, and can break other unrelated things.

Next time you think of using throw, instead, try doing an explicit return or an Either. Then unit test test it. Make it return an error in a larger program and see if it’s easier for you to find it given you are the one who caused it. I think you’ll find explicit returns or Eithers are easier to debug, easier to unit test, and can lead to more well thought out applications.

In my journey to learn functional programming and drink deep of the kool-aid, I wanted to share my latest learnings. Specifically around the quest for no errors and how you get things.

I’ve also just recently applied these same concepts in Python, not just JavaScript, so I’ll mix and match the examples to show how the concepts are universal.

After reading this article, you should understand why errors aren’t helpful embedded in your code & avoiding them is good and why we use functions to get things instead of the old way of assigning variables. Check out the first article if you missed it.

Recap of Pure Functions

If you’ve read my previous article, and hopefully Eric Elliot’s Pure Function one as well, you should know what pure functions are, and why are they are important in functional programming.

If not, a quick recap. Pure functions are functions that always give the same output given the same input.

function alwaysTrue()
{
   return true;
}

The function alwaysTrue will always return true no matter what you pass it, or don’t, or what order you call that function amongst other functions.

console.log(alwaysTrue()); // true
console.log(alwaysTrue('cow')); // true

This function is not pure:

function addStuff(a, b)
{
  return a + b + data;
}

If we call it with 1 and 1, we get an error:

var result = addStuff(1, 1); // goes boom

If we set data to 1, it gives us 3:

var data = 1;
var result = addStuff(1, 1); // 3

If we change data to 2, we get 4:

var data = 2;
var result = addStuff(1, 1); // 4

Finally, if we change data after we call it, and call it again, we get different results from both function calls:

var data = 1;
var result1 = addStuff(1, 1); // 3
data = 2;
var result2 = addStuff(1, 1); // 4

An impure function is not predictable, cannot be re-used without dragging a bunch of other variables and functions that make sure all the state is correct, and cannot be combined with other functions reliably because we don’t know how it will react.

Programming is supposed to be fun, so creating pure functions ensures what you make actually works and allows you to make bigger software programs that run (somewhat) predictably.

The Quest for No Errors

As you’ve seen above, errors make a function not pure. Some languages attempt to make this pure. Java has throwable and throws to ensure the code only compiles of the caller has ensured all possible error types are handled. Python, while dynamic, still has this philosophy of catching specific types of errors first, then unknowns after. These don’t catch all errors at compile time, however. Haskell uses Maybe to balance this with pure functions.

Runtimes errors arise from when you load data such as JSON from REST API’s, use 3rd party libraries that do not indicate they throw errors yet do, or you’re using dynamic languages which have little to no compiler help.

You have 2 choices: Know all potential errors, or don’t use errors.

I’ve talked in the past about not creating errors using exceptions, and instead indicating in an opt-in way of what went wrong for programmers consuming your code. The reason for this is usually all errors are unrecoverable. The best you can, and should, do is log it so you or others can see it and either know what’s going on, or debug the code and fix it later. Once you mix your code with other code, you’re part of the solution, not part of the problem.

If errors make the code less pure, then how do we not use errors?

How Do You Avoid Using Errors?

Let me demonstrate using the Amazon AWS Python API. They have an awesome API that’s mostly consistent. For Python, the library is called Boto3, and unlike JavaScript, Python is a blocking language so asynchronous code will stop on the line making asynchronous calls (like calling an AWS server) until it’s complete.

If you don’t know AWS, just know that an “EC2” or “ee-sea-two” is Amazon’s word for a “server”. You can easily create and destroy these things. You get instances of them, just like you get instances of an object in normal programming.

The following code searches my Amazon account for any servers that are owned by me. When I create new EC2’s, I tag them with metadata, in this case owner is my name. That way, if I have thousands of servers, I can easily find mine.

response = ec2.describe_instances(
    Filters=[
        {
            'Tag': 'owner',
            'Values': [
                'jesterxl',
            ]
        },
    ]
)

The above code isn’t pure. The Node API will follow the standard Node callback convention. The Boto3 Python API, in this case, either:

1. works and your response is a dictionary with search result information (JavaScript Object)
2. doesn’t work, and raises a Python Exception (JavaScript throw)
3. doesn’t work, and response isn’t a dictionary (rare, depends on API call such as S3.head_bucket)

Let’s make this pure by defining a pure Python function:

from pydash import is_dict

def are_my_servers_there(ec2):
    try:
        response = ec2.describe_instances(
            Filters=[
                {
                    'Tag': 'owner',
                    'Values': [
                        'jesterxl',
                    ]
                },
            ]
        )
        return is_dict(response)
    except Exception as e:
        print("find_my_servers failed:", e)
        return False

result = are_my_servers_there()
# result will always be True or False

This function always returns True or False. Here’s 2 example nose2 unit tests:

def test_works:
  value = are_my_servers_there(ec2_mock_good)
  self.assertTrue(value)

def test_fails:
  value = are_my_servers_there(ec2_mock_bad)
  self.assertFalse(value)

If the Python syntax is throwing you off, here is the equivalent JavaScript Node syntax for comparison:

const _ = require('lodash');
const areMyServersThere = (ec2, callback)=>
{
    try
    {
        var params = {
            Filters: [
                {
                    Name: 'owner',
                    Values: [
                        'jesterxl',
                    ]
                }
            ]
        };
        ec2.describeInstances(params, (err, data)=>
        {
            if(err)
            {
                console.log("describeInstances failed in callback:", err);
                callback(false);
                return;
            }
            callback(_.isObject(data));
        });
    }
    catch(boom)
    {
        console.log("describeInstances failed:", boom);
        callback(false);
    }
};
const result = findMyServers();
// result is always true or false

This function always calls the callback with true or false. And the Mocha/Chai tests:

it('should find my servers', (done)=>
{
  areMyServersThere(ec2MockGood, (foundThem)=>
  {
    if(foundThem)
    {
      done();
    }
    else
    {
      done(new Error("Couldn't find them.");
    }
  });
});
it('should not find my servers if not there', (done)=>
{
  areMyServersThere(ec2MockBad, (foundThem)=>
  {
    if(foundThem === false)
    {
      done();
    }
    else
    {
      done(new Error("Somehow succeeded when it shouldn't.");
    }
  });
});

If you search for Python code online, sometimes it will write except for a couple of types of Exceptions first, and then default to the generic one like I did last. This is often for clean up code in the case of database connections, file/folder shuffling, or networking handles. Low level stuff requires low-level error checking, heh!

Node, on the other hand, at least in API development still can’t escape try/catch. Inside of callbacks developers will parse data. Outside they’ll create it for requests. The call that accepts a callback may not have proper error handling internally and throws before it can call the callback. Global error handling is still a challenge, both in browsers and in Node.

The good news is not all functions you use throw. Some are pure by default (Lodash and Pydash for example). Some native API’s are predictable and only 1 or a few errors. Some are small predicates which you know won’t throw errors. In those cases, if you need to know about specific ones, you can handle those. How you go about that in a pure way, however, is up to you. As long as you call the same function with the same inputs and it produces the same outputs, then however you’re handling the exceptions is probably pure.

Things get fuzzy with I/O & databases. At that point you have to ask how pure do you want to make things? Easy answers are:

• If it works, and deadline is approaching, then pure enough.
• If it works, isn’t pure, and deadline is far away, then not pure enough.

Getting Things

Python and JavaScript throw errors when you attempt to access nested properties on an Object and misspell something, or the property simply isn’t there. This is lessened if you utilize a strongly typed language like TypeScript, and have good code editors that have intellisense and help you explore the Object before you type.

Property access is impure. Since you don’t have an ‘input’, there is no way to ensure you’re always getting the same result. This is used all the time in REST API’s where you get JSON back and start digging into the yummy data only to realize it’s missing things.

someRestCall()
.then((results)=>
{
  // should be results.data.user
    const user = results.user;
})
.catch((err)=>
{
  // either we failed to make the call,
  // or we failed to parse the data.
});

Another is lower-level API’s, such as detecting if you have the Performance API to detect browser refresh:

if(window.performance)
{
    if(performance.navigation.type  == 1 )
    {
        console.log('page reloaded');
    }
}

That code is all well in good in the latest browsers… except for Node where it throws an exception since she doesn’t have a window. You’ll run into this when you start using Babel extensions that were meant to only work for Node, but you want to test your browser code outside of Karma.

Another area is when you start combining functions and list comprehensions together for the above. Once you throw higher order functions together, while pure, you may not know exactly what you’re getting. This could just be too much cognitive load, you’re dealing with 3rd party impure code in the mix, or you’re dealing with I/O like asynchronous REST API’s. Even in this mostly pure world, you still can’t trust your Objects.

To state it clearly: Objects are not pure functions.

Lodash’ get is, though. Pydash has get as well.

In the above REST API, instead of accessing the response data impurely, we can use get:

const user = _.get(results, 'user');

… now when I first saw get, I thought it was stupid. I had no idea what pure functions were, nor was I using a lot of list comprehensions together. I’ve been accessing Objects for ages, why use a function?

Now you know why:

1. _.get is pure
2. it doesn’t throw errors
3. You can default what you want so if it’s not there, you always get a black and white result: it’s either there, or it ain’t.

For example, single dots on things you know exist are ok:

var result = {
    data: {
        user: {
            name: 'Jesse'
        }
    }
};

var user = result.user;
console.log("user:", user); // user: undefined

However, here we misspell data to dat, and it throws:

var result = {
    data: {
        user: {
            name: 'Jesse'
        }
    }
};

var user = result.dat.user;
console.log("user:", user); // never run

Watch when happens when we use _.get:

var user = _.get(result, 'dat.user');
console.log("user:", user); // user: undefined

You can even delete result:

var user = _.get(undefined, 'dat.user');
console.log("user:", user); // user: undefined

Outside of pure functions in the mutable state world, they can still save you from uncaught exceptions!

Combine that with it’s 3rd parameter which is a default value, and you can get even more predictable results:

expect(_.get(undefined, 'dat.user', undefined).to.equal(undefined);
expect(_.get(result, 'data.user.name', 'chicken')).to.not.equal('chicken');

We’ll see more of get‘s 3rd parameter when we start composing functions in a future article.

Et tu, Array?

(Arrays are full of holes… get it? lol!)

Even Arrays, a type of Object in JavaScript, are impure when you access them. This is part of the reason you see functions in Lodash for head, last, and tail (Although, tail is more for those coming from functional backgrounds where the first Array element is t3h special, a la Erlang/Elixir).

Here’s a common error:

var list;
result = list[2];
console.log("result:", result); // never runs

Solve that by using get instead:

var list;
result = _.get(list, '2');
console.log("result:", result); // undefined

A side benefit is that get works with Objects and Arrays so is flexible in that you can pass either.

Conclusions

The get function allows us a pure way to access Objects & Arrays in JavaScript and Dictionaries & Lists in Python. With reasonable defaults, you can use this to build and compose pure functions instead of littering them with impure property access.

Once you start composing functions like we’ll be doing in the next article, you want to ensure the main ones are as pure as possible. Debugging composed pure functions can be hard at first, so getting it right in the beginning helps.

You’ve also seen how errors are just that: something to be avoided instead of embraced. They make your functions impure, and confuse those who were expecting specific return values from your functions.

You do not need to use try/catch everywhere. In your own code, you’ll know when there could be areas which are out of your control and you need to. If using 3rd party code, if you have the time, see at what points they fail intentionally through unit and integration tests. This ensures you only use try/catch in areas you either don’t know (yet) or are intentional like in Python’s Boto3 library.

One important part of errors that shouldn’t be missed, however, is intentional ones. As you start creating higher level functions like the checkers in my previous article, giving good feedback is important. The reason(s) why predicates return false, or why composed functions return an error result instead of success help mitigate debugging, and ensure those learning your API know what’s really going on. Anyone who has used both React and Angular knows how superior React’s error messaging is.

Verbose, helpful errors are great and those SHOULD be embraced. However, you don’t have to raise/throw them unless it truly is an error that you want developers to know about it early.

Download – Keynote | PDF | JPEG’s | Slideshare

Resources Referenced:

• Steve Krug’s Book “Don’t Make Me Think”
• Gary Vaynerchuk’s Book “Crush It!”
• “Minimum Viable Product” blog post
• Startup Book “Start Small, Stay Small”

Announcements:

• D2W Conference
• Dee Sadler – Twitter | Site
• RIA Radio – Episode 8: Jesse Warden
• DebugWindow v2 – Log window for Flex apps

References:

• PyCon – Atlanta Python Conference
• Joel Hooks – Twitter | Blog
• Robotlegs – MVCS Framework for AS3 and Flex
• Nick Joyce on Twitter
• PyAMF – AMF support for Python
• Universal Mind – RIA Consulting Firm
• 

Beer – Milk Stout by Left Hand Brewing Company

]]> https://jessewarden.com/2010/02/roundtable-with-joel-hooks-nick-joyce-jxltv-episode-006.html/feed 3