Preface

I wrote this book for professional Java developers who would like to start working in Scala. You should already be able to program and know your way around Java’s classes, lambdas, streams and build tools. You don’t need to have seen any Scala before, or know anything about functional programming yet.

If you’re brand new to programming, this probably isn’t the book to start with. I’ve used Cats Effect when discussing effect systems. ZIO and Spark each get a section in appendix B to give a little flavour, but it would be a big book if I gave them the same treatment.

You’ll be typing and running code from the first few pages. I explain what happened after you’ve seen it work, and the names that theory has for these things come later, once you’ve used them. Everything the book prints as output came from running its own code, on Scala 3.9 and Java 25 unless otherwise noted.

The whole book builds one project, Fully Booked, a table-booking system for a single restaurant. In chapter 1 it’s a script. By chapter 22 it’s an HTTP service with a Postgres database, tests and a Docker image.

People write Scala in two broad styles, as a more concise Java or as a functional language, and most books stick to one of them. This one starts with the first and works its way to the second.

Part I, “Scala for Java developers”, stays close to Java and covers values, case classes, collections and testing. Part II, “Getting the compiler to do the work”, brings in functions as values, types that model the problem and errors as ordinary values. Part III, “Effects”, deals with programs that do input and output and run concurrently, using the Cats Effect library. Part IV, “A real service”, gives the project a build tool, an HTTP API, a database, more tests and a container. It ends with a chapter on Play, a framework that a lot of existing Scala is written in. The two appendices cover reading Scala 2 and the parts of the ecosystem the book doesn’t use.

Scala usually gives you more than one way to do something. I’ve picked one each time. Where that choice is contentious, I’ve said why I made it and what the alternative is.

You’ll need a computer with a terminal. I’ve written the commands for macOS and Linux. Scala runs on Windows as well, though a few of the commands will need translating or a WSL terminal. Chapter 1 installs Scala, and later chapters tell you what else to install when you need it. From chapter 20 that includes Docker.

It’s best to read the chapters in order, because each one builds on the code from the one before. Every chapter ends with a few exercises. You can download the code for every chapter, with solutions to most of the exercises, from scalafoundations.com. If you get stuck or want to skip ahead, the download has the project as each chapter left it.

Part I: Scala for Java developers

1. Up and running

The quickest way to get a feel for a language is to run some code, so we start by installing Scala and running a few small programs. After that I’ll describe two kinds of Scala codebase you could end up working in, which is what decides the order of this book.

1.1. Installing Scala

Scala runs on the JVM and needs JDK 17 or newer. Check what you have:

$ java -version

I ran the code in this book on JDK 25, the current long-term support release, but anything from 17 up will get you started.

The Scala project recommends installing through Coursier, its installer for Scala tools. Coursier’s command is cs, though Homebrew installs it under the longer name coursier. The setup command puts the scala command on your path, along with a JDK if it can’t find one already installed. On macOS with Homebrew:

$ brew install coursier && coursier setup

On Linux (x86-64):

$ curl -fL https://github.com/coursier/coursier/releases/latest/download/cs-x86_64-pc-linux.gz | gzip -d > cs
$ chmod +x cs && ./cs setup

Windows has an installer that you download and run. It’s linked from the official install page, https://docs.scala-lang.org/getting-started/install-scala.html, along with the commands for ARM machines.

Open a new terminal so that the changed path takes effect, then ask for the version:

$ scala version
Scala code runner version: 1.16.0
Scala version (default): 3.9.0

Your numbers will probably be higher than mine. Don’t worry about that, or about which JDK you have. Later in the chapter we add two lines to a source file saying which Scala and which JDK it wants, and the scala command downloads them if you don’t have them.

If you’ve looked at older tutorials, you may have seen them start by installing sbt, the main Scala build tool, and generating a project with it. You can skip all of that for now. Since Scala 3.5 the scala command has been a small build tool in its own right, one that compiles and runs your code and fetches whatever libraries the code asks for. The same program also goes by the name Scala CLI, and the “code runner version” in the output above is its version number. It’s all we need until chapter 18, where we move to sbt.

1.2. A first program

Make an empty directory to work in. I’ve called mine fully-booked, for reasons that will be clear in a few pages. In it, create a file called hello.scala:

@main def hello(): Unit =
  println("Hello from Scala")

Run it:

$ scala run hello.scala
Compiling project (Scala 3.9.0, JVM (25))
Compiled project (Scala 3.9.0, JVM (25))
Hello from Scala

The first run is slow while the tool downloads a compiler and starts a compile server in the background. After that, running an unchanged file takes under a second. The Compiling and Compiled lines appear whenever something had to be recompiled. I’ll leave them out of most of the book’s examples from here on.

There’s no class in that file. def defines a method, and @main marks this one as the program’s entry point. From that the compiler generates the class and the main(String[] args) that the JVM starts from. You can see the result under the .scala-build directory that has appeared next to your file, where there’s now a hello.class.

A method definition reads in a different order from Java’s: the name and parameters first, then a colon and the return type, then an = and the body. Unit is Scala’s equivalent of void. println needs neither an import nor System.out in front of it. It’s one of a few methods that are available everywhere.

There are no braces around the body either. As in Python, the body is whatever is indented under the def. Try putting the braces back, with a semicolon for good measure:

@main def hello(): Unit = {
  println("Hello from Scala");
}

That runs as well. Braces are still legal, and code written before Scala 3 is full of them. I use the indented style in this book, as the official Scala 3 documentation does.

1.3. The REPL

Scala has an interactive prompt, known as the REPL (read, evaluate, print, loop). Start it with scala repl. It prints a welcome line and then waits at a scala> prompt. The numbers in the list below are the seats at each of a restaurant’s six tables, which we come back to in the next section. Type these in:

scala> 2 + 3
val res0: Int = 5

scala> val seats = List(2, 2, 4, 4, 6, 8)
val seats: List[Int] = List(2, 2, 4, 4, 6, 8)

scala> seats.sum
val res1: Int = 26

scala> seats.filter(n => n >= 4)
val res2: List[Int] = List(4, 4, 6, 8)

scala> "table for two".toUpperCase
val res3: String = "TABLE FOR TWO"

scala> seats.sum / seats.size
val res4: Int = 4

scala> res1 + 1
val res5: Int = 27

scala> seats = List(10)
-- [E052] Type Error: ------------------------------------------------------------------------------
1 |seats = List(10)
  |^^^^^^^^^^^^^^^^
  |Reassignment to val seats
  |
  | longer explanation available when compiling with `-explain`
1 error found

Apart from the last one, each reply has the form val name: Type = value. When you defined seats, you never said it was a List[Int], which is how Scala writes Java’s List<Integer>. The compiler worked that out from the right-hand side. This is type inference. Java’s var does it for local variables, but Scala does it almost everywhere, so Scala code has far fewer type names in it than Java code does. It’s still statically typed, and the types the REPL prints are fixed at compile time. The session ends with a deliberate failure. seats is a val, Scala’s version of a final variable. Trying to reassign it is a compile error, complete with a position and an error code. Scala’s own var keyword is the counterpart of val: it declares a variable that can be reassigned. Types are inferred for both.

List(2, 2, 4, 4, 6, 8) builds a list, with no new and no List.of. Calls such as seats.sum and seats.size are method calls, not field accesses. Scala methods that take no arguments and simply return a value are usually declared and called without parentheses, and Java’s no-argument methods can be called that way too. By convention the parentheses stay on methods that do something, which is why hello() has them.

Most of the rest works the way it does in Java. A Scala string is a Java string, making "table for two".toUpperCase the toUpperCase() that java.lang.String has in Java. Dividing one Int by another truncates, hence the 4 from seats.sum / seats.size. Every Java library is available in the same way, your company’s internal jars and every JDBC driver included. The argument to filter is a lambda, which Scala writes with => where Java uses ->.

Results that you didn’t name get names anyway (res0, res1 and so on). The res1 + 1 line uses one. Type res2. and press Tab to see what else a List can do.

Leave the REPL with :quit or Ctrl-D. I’d keep one open in a spare terminal while you read, though. If something in this book looks wrong or surprising, type it in and see.

1.4. Something closer to real work

From here on, the code in this book belongs to one project: Fully Booked, a booking system for a single restaurant. I picked restaurant bookings because the rules need no explaining. A table can’t hold two parties at once, and a party of six doesn’t fit at a table for two.

Its first program finds a table. Given the size of a party, it picks the smallest of the restaurant’s six tables that they’ll fit at. Save it as tables.scala:

case class Table(number: Int, seats: Int)

val tables = List(
  Table(1, 2), Table(2, 2), Table(3, 4),
  Table(4, 4), Table(5, 6), Table(6, 8)
)

@main def findTable(partySize: Int): Unit =
  val bigEnough = tables.filter(table => table.seats >= partySize)
  if bigEnough.isEmpty then
    println(s"Sorry, nothing seats $partySize")
  else
    val best = bigEnough.minBy(table => table.seats)
    println(s"Table ${best.number} (seats ${best.seats})")

Arguments for your program go after a --, which separates them from arguments meant for the scala command itself:

$ scala run tables.scala -- 3
Table 3 (seats 4)
$ scala run tables.scala -- 5
Table 5 (seats 6)
$ scala run tables.scala -- 12
Sorry, nothing seats 12

Read tables.scala from the top as if it were Java, and most of it makes sense. case class Table(number: Int, seats: Int) is close to a Java record: one line declares a class with two fields, plus the constructor, equals, hashCode and toString that go with them. You create a Table without new. The list of tables is a val at the top level of the file, outside any class, with its type inferred as List[Table].

findTable is where it stops looking like Java. filter is called directly on the list, with another lambda, and hands back a new List. No .stream() goes in front and no .toList() behind. The if has a then and no parentheses. minBy picks the element for which its lambda returns the lowest value, here the table with the fewest seats. In the strings with an s in front, $partySize is replaced by the variable’s value and ${best.number} by the value of a whole expression.

minBy throws an exception on an empty list, and the only thing preventing that here is the isEmpty check above it, which would be easy to leave out. Chapter 5 has a better way to handle an answer that might be “nothing”.

Now try to misuse the program. findTable declares its parameter as partySize: Int, so the entry point that @main generates has to turn a command-line string into an integer. Leave the argument off, then pass a word where the number should be:

$ scala run tables.scala
Illegal command line: more arguments expected
$ scala run tables.scala -- abc
Illegal command line: java.lang.NumberFormatException: For input string: "abc"

1.4.1. The same program in Java

For comparison, this is how I’d write it in Java 25:

record Table(int number, int seats) {}

List<Table> tables = List.of(
    new Table(1, 2), new Table(2, 2), new Table(3, 4),
    new Table(4, 4), new Table(5, 6), new Table(6, 8));

void main(String[] args) {
    int partySize = Integer.parseInt(args[0]);
    var best = tables.stream()
        .filter(table -> table.seats() >= partySize)
        .min(Comparator.comparingInt(Table::seats));
    if (best.isEmpty()) {
        IO.println("Sorry, nothing seats " + partySize);
    } else {
        Table table = best.get();
        IO.println("Table " + table.number() + " (seats " + table.seats() + ")");
    }
}

If your JDK is version 25, java FindTable.java 3 runs it and prints the same answer. On an older JDK it won’t compile, because it relies on things that arrived in Java 25. A small program like this one no longer needs a class around its methods and fields, public static on its main, or imports for List and Comparator. IO.println is new as well.

That’s 18 lines against 14, and what’s left over is things like new, .stream(), Integer.parseInt and a Comparator. It’s clutter, but you probably stopped noticing it years ago. Java has improved a great deal since lambdas and records arrived, and I wouldn’t learn a new language to save four lines. The differences that do justify it begin in chapter 4, with collections.

1.5. Breaking it on purpose

You’re going to read a lot of Scala compiler errors, so let’s cause one. In tables.scala, change table.seats on the filter line to table.seat. Run the program again:

$ scala run tables.scala -- 3
[error] ./tables.scala:9:42
[error] value seat is not a member of Table - did you mean table.seats?
[error]   val bigEnough = tables.filter(table => table.seat >= partySize)
[error]                                          ^^^^^^^^^^
Compilation failed

Put the s back.

Scala’s compiler checks far more than javac does. Much of this book is about arranging code so that a mistake shows up like this, on your screen, where Java might have given you a NullPointerException in production.

This error was easy to read, but once generic types and library machinery get involved an error can run to a dozen lines. I’ll go through the nasty ones as we hit them.

1.6. Pinning versions

I said earlier that it doesn’t matter if your Scala version is newer than mine. Add these two lines at the very top of tables.scala:

//> using scala 3.9.0
//> using jvm 25

These are directives. The compiler sees them as comments, but the scala command reads them before it compiles anything and treats them as build settings, in this case which version of Scala to compile with and which JDK to run on. If you don’t have that compiler or that JDK, it downloads them. A file that starts with these two lines builds the same way on your laptop as it does on mine.

Libraries are declared in the same way, with //> using dep followed by the library’s Maven coordinates, in place of a pom.xml or build.gradle.

I’ve pinned 3.9 because it’s the current LTS (long-term support) release. Scala 3 has two kinds of release. Most libraries are built against an LTS version, because those get bug fixes for years. 3.9 became the LTS in September 2026. Between LTS versions come “Scala Next” releases a few times a year with newer features, 3.10 being the latest as I write. At work, use the LTS unless you need something specific from a newer release.

1.7. An editor

A plain text editor is enough for this chapter. From the next one onwards you’ll want errors shown as you type, plus the ability to hover over a name and see the type the compiler inferred for it. That means VS Code or IntelliJ.

In VS Code, install the Metals extension. Metals is Scala’s language server, the background program that an editor asks for completions, errors and types. Because it gets its errors and types from the real compiler, your editor never disagrees with your build about whether the code compiles. The same server works in Neovim, Zed, Emacs and anything else that speaks the Language Server Protocol.

If you write Java for a living you probably have IntelliJ IDEA open already, and with the Scala plugin installed you can stay there, keeping the refactoring tools and the debugger you’re used to. It also copes well with a project that mixes Java and Scala. For years its weakness was a type checker of its own, which sometimes marked valid Scala 3 code as an error, but current versions highlight Scala 3 using the compiler by default and that complaint has mostly gone away.

Whichever you pick, run scala setup-ide . in your working directory before you open that directory in the editor. The command writes a small .bsp directory that tells the editor how the code is built.

If you have no preference, use VS Code with Metals.

1.8. Two kinds of Scala codebase

When somebody says they work on a Scala codebase, they could mean one of two quite different things. The first is written as a more concise Java. You’ll find classes and traits (traits are Scala’s interfaces), a dependency injection container, exceptions for errors and Future (Scala’s counterpart to CompletableFuture) for anything asynchronous, with mutable fields wherever they’re convenient. Usually it sits on a large framework: Play if it’s a web application, Akka (or Pekko, the fork made when Akka stopped being open source) if it’s built around actors (objects that communicate only by sending each other messages) and streaming. Data pipelines written for Apache Spark, the cluster computing engine, tend to look like this too. A Java developer can usually find their way around such code quickly. A great deal of the Scala written in the 2010s is of this kind, and much of it is still running and still being hired for.

The second is written in the functional style. To change a booking, this code builds a new one with the difference applied and leaves the old one alone. A method that can fail hands the failure back to its caller as an ordinary return value, where Java would throw. The part that takes the most getting used to is how such code talks to the outside world: a database query or an HTTP call is wrapped in a type called IO and run by a library known as an effect system, of which the two big ones are Cats Effect and ZIO. It’ll probably look strange to you at first, with hardly a var in sight and method signatures like IO[Either[BookingError, Booking]].

It’s all one language and the two kinds of code can call each other, but their libraries barely overlap. Developers who prefer one kind tend to hold firm views about the other.

This book has four parts and covers both kinds, the more concise Java first. Part I is the first kind, Scala as a Java developer tends to write it at first. By the end of it you should be able to read and change code in that sort of codebase. Parts II and III work towards the functional style one piece at a time. Wherever a library supplies some clever abstraction, we write a small version of it ourselves before using the library’s. Part IV builds a complete service that way. Then the final chapter rebuilds a slice of it in Play, since that could well be what your next employer runs.

I take the book as far as the functional style because I think it’s the reason to pick Scala in the first place. If what you want is Java with less typing, Kotlin gives you that with better tooling and less to learn. Scala’s advantage over Java and Kotlin is a type system, and a set of libraries built on it, that let the compiler check things other languages leave to tests and code review, such as whether every way a request can fail has been handled. The same libraries make concurrent code much easier to get right.

1.9. The downsides

Compiling is noticeably slower than it is for Java, to the point where a large project can take minutes to build from clean. Incremental compilation hides most of that while you’re working, but CI builds are long.

You’ll also find far fewer Scala jobs than Java jobs. In the six months to November 2026, about one in two hundred adverts for IT contract work in the UK mentioned Scala.

Scala 3 arrived in 2021 with enough changes that migrating a big codebase from Scala 2 is a project in itself, one that many companies still haven’t started. Expect to meet Scala 2 in production. I teach Scala 3 only, with appendix A as a guide to reading the older language when you run into it.

Scala’s libraries also come in rival camps that don’t combine well, so a team has to pick one. Typelevel is a group of projects built around Cats Effect, among them the HTTP library http4s and the database library doobie. ZIO has its own equivalents of each. Akka and Pekko come with an HTTP server and streaming of their own. Li Haoyi has written a set of deliberately simple libraries (Cask for HTTP, uPickle for JSON, and others) that avoid effect systems altogether. This book uses Typelevel. If you end up at a ZIO company instead, most of what you learn here carries over.

1.10. Where Fully Booked goes from here

tables.scala is as small as Fully Booked gets. A real booking system has awkward cases to deal with. Two people ask at the same moment for the last table free at eight o’clock, and exactly one of them should get it. A booking that was cancelled an hour ago must not be seated.

Over the course of the book, Fully Booked grows from scripts like tables.scala into a tested HTTP service, backed by Postgres and packaged as a Docker image. The code for every chapter can be downloaded from scalafoundations.com. If you skip a chapter, the download has the code as that chapter left it.

1.11. Exercises

Solutions are in the book’s downloadable code, under ch01/exercises.

  1. Run tables.scala for a party of 2. Tables 1 and 2 both have two seats. Which one do you get, and what does that tell you about how minBy treats a tie?

  2. Add a table 7 with ten seats, and check that a party of 9 is given it.

  3. Start the REPL with scala repl tables.scala, which loads the file first so that tables is available at the prompt. Work out the total number of seats in the restaurant. Then find how many tables seat four or more. Type tables. and press Tab to see the methods a List has; map, sum and count are the ones you need. map works as it does on a Java stream, and count takes a lambda as filter does.

  4. Change the program so that it prints every table the party would fit at, smallest first, instead of only the best one. sortBy takes the same kind of lambda as minBy, and foreach runs a lambda once for each element.

  5. Give findTable a second parameter, name: String, and include the name in the output. Then run it with too few arguments.

2. Values, methods and expressions

Before Fully Booked takes a booking, it has to be able to tell the customer what they’re agreeing to. Two things matter to them: how long they get the table for, and whether there’s a deposit to pay. This restaurant’s rules are:

  • A party of up to four has the table for 90 minutes. Bigger parties get two hours.

  • Parties of six or more pay a deposit of £10 a head.

We’ll turn those rules into a small program, quote.scala, writing it first the way a Java developer might on their first day with Scala and then improving it a step at a time. Most of Scala’s everyday syntax turns up along the way.

2.1. A method, written the Java way

Create quote.scala in your working directory:

//> using scala 3.9.0
//> using jvm 25

def stayMinutes(partySize: Int): Int =
  var minutes = 0
  if partySize <= 4 then
    minutes = 90
  else
    minutes = 120
  return minutes

@main def printQuote(partySize: Int): Unit =
  val minutes = stayMinutes(partySize)
  println(s"Party of $partySize: table for $minutes minutes")

Run it with --watch this time:

$ scala run quote.scala --watch -- 5
Party of 5: table for 120 minutes
Program exited with return code 0.
Watching sources, press Ctrl+C to exit, or press Enter to re-run.

With --watch the command doesn’t exit. Each time you save the file, it recompiles and runs the program again. That suits a chapter like this one, where the same file keeps changing. I’ll show plain scala run commands from here on. If you like the watcher, keep it going in a terminal of its own and restart it whenever the program’s arguments change.

stayMinutes sits at the top level of the file, as tables did in chapter 1. It’s correct, though I wrote it as Java with different punctuation, right down to the variable that exists only so that each branch has something to assign to.

Scala source has no split between int and Integer: you write Int everywhere. The compiler uses a primitive int wherever it can and boxes where it must, inside a List[Int] for instance. Long, Double and Boolean get the same treatment.

2.2. Everything has a value

In Java, if is a statement: it does things but doesn’t produce a result, which is why Java also has the ?: operator. A Scala if is an expression. It evaluates to the value of whichever branch was taken, so the result can be assigned directly:

def stayMinutes(partySize: Int): Int =
  val minutes = if partySize <= 4 then 90 else 120
  return minutes

Now minutes is a name for a value that gets used once, on the very next line. Drop it. Drop the return as well. A method’s result is the value of the last expression in its body:

def stayMinutes(partySize: Int): Int =
  if partySize <= 4 then 90 else 120

Save the file and run it again. It prints the same answer as before.

This is what Scala methods normally look like. You’ll rarely see return in one (the keyword exists, but it’s seldom needed when every if produces a value). You won’t see a ternary operator at all, since if with an else already is one.

Blocks follow the same rule: any indented run of lines is an expression, whose value is the value of its last line. That means a block can go anywhere a value can. Inside stayMinutes, for instance, you could write:

val minutes =
  val small = partySize <= 4
  if small then 90 else 120

small exists only inside the block. A block like this is a tidy way to stop a helper value leaking into the rest of a method.

If everything has a value then so does println("hello"). Its value is (), which is the only value of the type Unit and represents “nothing”. An if without an else produces () as well. Where void is a special case in Java’s grammar, Unit is an ordinary type, which is why printQuote can declare it as a return type like any other.

2.2.1. val or var?

Developers in both kinds of codebase from chapter 1 generally use val unless they have a reason not to. The argument is the one for final in Java: if a name can’t be reassigned, you never have to read the rest of a method to find out what it refers to further down. Few people write final on Java locals, probably because it’s extra typing. In Scala, val is no more typing than var.

A var can change its value but never its type:

scala> var booked = 0
var booked: Int = 0

scala> booked = booked + 2
booked: Int = 2

scala> booked = "two"
-- [E007] Type Mismatch Error: ---------------------------------------------------------------------
1 |booked = "two"
  |         ^^^^^
  |         Found:    ("two" : String)
  |         Required: Int
  |
  | longer explanation available when compiling with `-explain`
1 error found

A var that’s only used inside a single method, such as a running total in a loop, does no harm. The ones that cause trouble are var fields on objects that several threads can reach, for the same reasons as in Java. Either way, expect a reviewer to ask about any var.

2.3. Arrival times: using a Java library

A customer would sooner hear “until 21:30” than “120 minutes”, and for that the program has to do arithmetic on times. Scala has no date and time library of its own, and doesn’t need one, because java.time can be used directly. Try it in the REPL:

scala> import java.time.LocalTime

scala> val arrival = LocalTime.parse("19:30")
val arrival: LocalTime = 19:30

scala> arrival.plusMinutes(90)
val res0: LocalTime = 21:00

scala> arrival.isBefore(LocalTime.NOON)
val res1: Boolean = false

Apart from val and the missing semicolons, this is Java. Static methods and constants such as LocalTime.parse and LocalTime.NOON are used in the usual way. import can also take a whole package, with import java.time.*, or a few named classes, with import java.time.{LocalDate, LocalTime}.

Here’s the whole file with arrival times added. The time arrives as a string and is parsed inside printQuote, because @main can only convert simple types such as Int for you.

//> using scala 3.9.0
//> using jvm 25

import java.time.LocalTime

def stayMinutes(partySize: Int): Int =
  if partySize <= 4 then 90 else 120

@main def printQuote(partySize: Int, time: String): Unit =
  val arrival = LocalTime.parse(time)
  val leaveBy = arrival.plusMinutes(stayMinutes(partySize))
  println(s"Party of $partySize at $arrival: table until $leaveBy")

Run it:

$ scala run quote.scala -- 5 19:30
Party of 5 at 19:30: table until 21:30

Now give it a time that LocalTime.parse can’t read:

$ scala run quote.scala -- 5 7pm
Exception in thread "main" java.time.format.DateTimeParseException: Text '7pm' could not be parsed at index 0
	at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2108)
	at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:2010)
	at java.base/java.time.LocalTime.parse(LocalTime.java:475)
	at java.base/java.time.LocalTime.parse(LocalTime.java:460)
	at quote$package$.printQuote(quote.scala:10)
	at printQuote.main(quote.scala:9)

That’s an ordinary Java exception, with our own file in the bottom two frames of the trace. Scala has try, catch and finally to deal with one (a try is an expression, just as an if is). It doesn’t have checked exceptions. LocalTime.parse happens to throw an unchecked one, but you can equally call a Java method that declares throws IOException without writing a try or a throws clause of your own. That’s convenient until something throws that you weren’t expecting, since the compiler won’t have warned you. Chapter 9 shows how the functional style deals with this, by handing a foreseeable failure such as bad input back as a value.

2.4. Leaving types out, and putting them in

val arrival = LocalTime.parse(time) doesn’t say what type arrival is. The compiler inferred it, as it did in the REPL. Method parameters never get that treatment: their types must always be written out. Return types are in between: the compiler can infer them, but you’re allowed to state them. stayMinutes would compile without its : Int.

I’d keep it, and the quickest way to show why is to make a mistake on purpose. Delete the return type from stayMinutes (the : Int after the parameter list), then put quotation marks around the 120 to turn it into a string:

$ scala run quote.scala -- 5 19:30
[error] ./quote.scala:11:37
[error] Found:    Int | String
[error] Required: Long
[error]   val leaveBy = arrival.plusMinutes(stayMinutes(partySize))
[error]                                     ^^^^^^^^^^^^^^^^^^^^^^
Compilation failed

The compiler reports the error on line 11, in printQuote, at the call to stayMinutes. The mistake is on line 7. The message says it found an Int | String, which is how Scala writes a type that’s either an Int or a String. With no return type declared, the compiler decided that a method returning a number from one branch and a string from the other must be what you wanted. It only objected when plusMinutes, which takes a Long, was handed something that could be a String. In a real codebase that call could be in another file, written by somebody else.

Put the : Int back and leave the quotation marks in:

$ scala run quote.scala -- 5 19:30
[error] ./quote.scala:7:34
[error] Found:    ("120" : String)
[error] Required: Int
[error]   if partySize <= 4 then 90 else "120"
[error]                                  ^^^^^
Compilation failed

The rule I follow is to let the compiler infer the types of local values and to declare the return type of every method that isn’t a tiny private helper. The official Scala style guide asks for the same on all public methods. Besides putting errors in the right place, a declared type lets a reader see what a method returns without reading its body.

Take the quotation marks out again before going on.

2.5. Deposits: money and operators

The second rule involves money. Don’t keep it in a Double, which goes wrong in Scala exactly as it does in Java:

scala> 0.1 + 0.2
val res0: Double = 0.30000000000000004

scala> BigDecimal("0.10") + BigDecimal("0.20")
val res1: BigDecimal = 0.30

scala> BigDecimal("10.00") * 7
val res2: BigDecimal = 70.00

scala> BigDecimal("10.00").*(7)
val res3: BigDecimal = 70.00

The right type is BigDecimal, as it would be in Java, except that this is Scala’s own BigDecimal, a thin wrapper around java.math.BigDecimal whose main purpose is to let you add and multiply with + and *. If you’ve ever written price.multiply(quantity).add(delivery), you’ll see the appeal.

The * in that session is a method on BigDecimal that happens to have a symbol for a name. The last line of the session calls it the long way round, as BigDecimal("10.00").*(7). All of Scala’s operators work like this, including the ones on numbers: 2 + 3 is a call to a method named + on Int, though the compiler turns it into the same bytecode as Java’s addition.

Library authors sometimes get carried away with this. You’ll meet methods called <+> and *>, which you can neither pronounce nor type into a search engine. Good Scala code keeps symbols for things that are operators everywhere else, mostly arithmetic, and gives everything else a name. Where a later chapter needs one of the odd ones, I’ll tell you how to say it.

Here’s the deposit rule. Add it under stayMinutes:

def deposit(partySize: Int, perHead: BigDecimal = BigDecimal("10.00")): BigDecimal =
  if partySize >= 6 then perHead * partySize else BigDecimal(0)

perHead has a default value, so a call can leave it out. Load the file into the REPL with scala repl quote.scala and call the method a few different ways:

scala> deposit(8)
val res0: BigDecimal = 80.00

scala> deposit(8, BigDecimal("15.00"))
val res1: BigDecimal = 120.00

scala> deposit(perHead = BigDecimal("15.00"), partySize = 8)
val res2: BigDecimal = 120.00

scala> deposit(4)
val res3: BigDecimal = 0

Arguments can also be passed by name, in any order, which is what the call with perHead first is doing. Naming is most useful when a call would otherwise be a row of bare literals. deposit(8, BigDecimal("15.00")) is clear enough, but a reader of resize(800, 600, true, false) can’t tell what the two Booleans mean. Java has neither default nor named arguments, and that’s where its chains of overloads and its builder classes come from.

Now use deposit in printQuote:

@main def printQuote(partySize: Int, time: String): Unit =
  val arrival = LocalTime.parse(time)
  val leaveBy = arrival.plusMinutes(stayMinutes(partySize))
  val due = deposit(partySize)
  println(s"Party of $partySize at $arrival: table until $leaveBy, deposit $due")
$ scala run quote.scala -- 7 19:30
Party of 7 at 19:30: table until 21:30, deposit 70.00
$ scala run quote.scala -- 2 19:30
Party of 2 at 19:30: table until 21:00, deposit 0

The sums are right. The presentation needs work: there’s no pound sign, and “deposit 0” isn’t something to show a customer. Fixing it means testing whether a BigDecimal is zero, so I want to look at how Scala compares things first.

2.6. == means equals

Java developers learn early not to compare objects with ==. In Scala that’s how you’re meant to compare them:

scala> val a = new String("table")
val a: String = "table"

scala> val b = new String("table")
val b: String = "table"

scala> a == b
val res0: Boolean = true

scala> a eq b
val res1: Boolean = false

scala> BigDecimal("10.0") == BigDecimal("10.00")
val res2: Boolean = true

scala> new java.math.BigDecimal("10.0").equals(new java.math.BigDecimal("10.00"))
val res3: Boolean = false

scala> BigDecimal("70.00") == 70
val res4: Boolean = true

In Scala, == calls equals, so a and b are equal even though they’re separate objects. I used new, which Scala still accepts, to make sure of that. The old Java bug of comparing strings with == can’t be written. Reference comparison is still available as eq, a method that’s written between its operands the way == is, for the rare occasions when you want it.

Java’s BigDecimal.equals compares the scale as well as the value, making 10.0 and 10.00 unequal, and plenty of Java developers have been caught out by that. Scala’s wrapper compares by value alone. It even compares a BigDecimal with an Int sensibly. Comparing the deposit with 0 is all the zero test we need.

2.7. Tidying the output

printQuote is getting long, and working out a quote is a different job from printing one. I want a method that works the quote out and hands it back, which means returning two values, the leaving time and the deposit. In Java you’d write a small record to carry them. For a case this small, Scala has something lighter called a tuple:

scala> val pair = (90, "no deposit")
val pair: (Int, String) = (90, "no deposit")

scala> pair._1
val res0: Int = 90

scala> val (minutes, text) = pair
val minutes: Int = 90
val text: String = "no deposit"

scala> val named = (minutes = 90, text = "no deposit")
val named: (minutes : Int, text : String) = (90, "no deposit")

scala> named.text
val res1: String = "no deposit"

A tuple is a fixed number of values grouped together, each with its own type. (Int, String) is the type of a pair holding an Int followed by a String. You can get at the parts by position with ._1 and ._2, but that reads badly. Taking the whole tuple apart in one go, the way val (minutes, text) = pair does, is better.

named is a named tuple, where each part has a name as well as a position. Named tuples only arrived in Scala 3.7, so you won’t find them in codebases that are still on the previous LTS, 3.3.

Replace printQuote with these three methods:

def quote(partySize: Int, arrival: LocalTime): (leaveBy: LocalTime, deposit: BigDecimal) =
  val leaveBy = arrival.plusMinutes(stayMinutes(partySize))
  (leaveBy, deposit(partySize))

def describe(partySize: Int, arrival: LocalTime): String =
  val q = quote(partySize, arrival)
  val depositText =
    if q.deposit == 0 then "no deposit" else f"deposit £${q.deposit}%.2f"
  s"Party of $partySize at $arrival: table until ${q.leaveBy}, $depositText"

@main def printQuote(partySize: Int, time: String): Unit =
  println(describe(partySize, LocalTime.parse(time)))
$ scala run quote.scala -- 7 19:30
Party of 7 at 19:30: table until 21:30, deposit £70.00
$ scala run quote.scala -- 2 12:15
Party of 2 at 12:15: table until 13:45, no deposit

quote returns its two results as a named tuple, written as an ordinary pair in parentheses on its last line. describe reads them back out by name.

The one new piece of syntax is the f in front of the deposit string. An f string is an s string that also accepts printf formats after each value. Here %.2f prints the deposit to two decimal places.

Keep tuples for passing a couple of values from one method to the next inside a file. Once a group of values starts travelling around a codebase, or grows a third and fourth member, it should be a case class. Chapter 3 starts with those.

2.8. Every party size at once

It would be easier to check the rules if we could see quotes for a whole range of party sizes together. Replace printQuote with printQuotes, which takes only the time:

@main def printQuotes(time: String): Unit =
  val arrival = LocalTime.parse(time)
  for partySize <- 1 to 8 do
    println(describe(partySize, arrival))
$ scala run quote.scala -- 19:30
Party of 1 at 19:30: table until 21:00, no deposit
Party of 2 at 19:30: table until 21:00, no deposit
Party of 3 at 19:30: table until 21:00, no deposit
Party of 4 at 19:30: table until 21:00, no deposit
Party of 5 at 19:30: table until 21:30, no deposit
Party of 6 at 19:30: table until 21:30, deposit £60.00
Party of 7 at 19:30: table until 21:30, deposit £70.00
Party of 8 at 19:30: table until 21:30, deposit £80.00

The loop runs its body once for each number from 1 to 8 inclusive. Read the <- as “taken from”. This is the only kind of for in Scala, which has never had the for (int i = 0; i < n; i++) form. 1 to 8 builds a range, and since operators are methods you might guess that to is an ordinary method being called on 1. It is: 1 to 8 means 1.to(8). Write 1 until 8 if you want to stop at 7. Scala has while loops as well, without the do/while form and without break or continue keywords.

You’ll probably write fewer loops in Scala than you do in Java. Most of the jobs a Java for loop does are done by methods on collections, like the filter and minBy in chapter 1.

2.9. Indentation, braces and end

By now you’ve seen a fair amount of code in which indentation decides what belongs to what. If whitespace in Python or YAML has ever bitten you, you’ll want to know how far to trust it here.

The compiler is strict about indentation that doesn’t line up with anything. Here’s a version of stayMinutes with the else one space too far in, in a file I’ve called stay.scala, and what the compiler makes of it:

def stayMinutes(partySize: Int): Int =
  if partySize <= 4 then
    90
   else
    120
$ scala compile stay.scala
[error] ./stay.scala:4:4
[error] The start of this line does not match any of the previous indentation widths.
[error] Indentation width of current line : 3 spaces
[error] This falls between previous widths: 2 spaces and 4 spaces
[error]    else
[error]    ^
Compilation failed

scala compile builds a file without running it. A line that’s indented consistently but at the wrong level gets past any compiler, as in this second throwaway file, deposit.scala:

@main def askForDeposit(due: Int): Unit =
  if due > 0 then
    println("Deposit required")
  println("Card details needed")

The second println was supposed to be inside the if. It isn’t, so it runs for a customer who owes nothing:

$ scala run deposit.scala -- 0
Card details needed

Java has the same trap when an if is written without braces. I’d rate the risk about the same: low, because the mistake is at least visible on the screen, though it still gets through review now and then.

When a body is long enough that its first line has scrolled off the screen, you can mark where it finishes with end:

def stayMinutes(partySize: Int): Int =
  if partySize <= 4 then
    90
  else
    120
  end if
end stayMinutes

end markers are optional but checked: an end stayMinutes that closes something other than stayMinutes is a compile error. I’d put them on anything longer than a screen.

Braces remain legal everywhere, as you saw in chapter 1, and plenty of teams use them, particularly teams whose code began in Scala 2. Go with whatever the codebase in front of you already does. A team that changes its mind can have the compiler do the conversion. Put //> using options -rewrite -no-indent at the top of a file like ours and compile it once, and the compiler rewrites the file with braces. -rewrite -indent takes them out again.

Indent with two spaces. Scalafmt, the standard formatter, does the same, and so does every Scala codebase you’re likely to meet.

2.10. Exercises

Solutions are in the book’s downloadable code, under ch02/exercises.

  1. Lunch is quicker. Any party arriving before 15:00 gets 75 minutes, whatever its size. stayMinutes will need the arrival time as a second parameter. LocalTime.of(15, 0) gives you the cut-off, and conditions can be chained with else if.

  2. The owner decides that no deposit should be more than £100. BigDecimal has a min method. Check your change in the REPL with deposit(14).

  3. The kitchen takes bookings from noon until 21:30. Write kitchenOpen(arrival: LocalTime): Boolean, then make describe return a different message when the kitchen is closed.

  4. Predict the value and the type of each of these, then check in the REPL: "8" + 8, 8 / 3 * 3.0, BigDecimal("1.10") == BigDecimal("1.1") and (1, "a") == (1, "a").

  5. With quote.scala loaded in the REPL, work out the deposit for a party of 6 at £12.50 a head. Pass the arguments in the opposite order to the one in which deposit declares them.

3. Case classes, enums and traits

In chapter 2 a quote was held in a tuple, with a note that values which travel around a program together belong in a case class. A booking is that sort of value. It has a party, a table and a time, and later in the chapter it gets a status too. We’ll give each of those a type and write the code that stores and displays bookings, ending up with a small project of several files.

3.1. A booking as data

The fully-booked directory is about to become one program, with all of its files compiled together, and the scripts from chapters 1 and 2 would get in the way: tables.scala has its own Table, and each script has an @main that has nothing to do with the new code. Delete hello.scala, tables.scala and quote.scala, along with anything else you saved while trying things out, or move them to another directory if you’d like to keep them.

Then create a new file called model.scala:

//> using scala 3.9.0
//> using jvm 25

import java.time.LocalTime

case class Table(number: Int, seats: Int)

case class Party(name: String, size: Int)

case class Booking(party: Party, table: Table, time: LocalTime)

Table is the case class from chapter 1. The other two are new, with a Booking holding a Party, a Table and a LocalTime. Load the file into the REPL with scala repl model.scala and make a booking. Because the REPL doesn’t see the file’s own imports, the session starts by importing LocalTime again. LocalTime.of(19, 30) is half past seven in the evening.

scala> import java.time.LocalTime

scala> val booking = Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30))
val booking: Booking = Booking(
  party = Party(name = "Okonkwo", size = 5),
  table = Table(number = 5, seats = 6),
  time = 19:30
)

scala> booking.party.name
val res0: String = "Okonkwo"

scala> booking == Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30))
val res1: Boolean = true

scala> val later = booking.copy(time = LocalTime.of(20, 0))
val later: Booking = Booking(
  party = Party(name = "Okonkwo", size = 5),
  table = Table(number = 5, seats = 6),
  time = 20:00
)

scala> booking.time
val res2: LocalTime = 19:30

scala> booking.toString
val res3: String = "Booking(Party(Okonkwo,5),Table(5,6),19:30)"

Laying a case class out with its field names is the REPL’s own way of displaying one. The toString that the compiler generated, which is what println uses, is the more compact text on the last line. The compiler generated equals as well, so two bookings built separately from the same values are ==.

Because the fields of a case class are all vals, a booking can’t be altered once it exists. If the Okonkwos ring to say they’ll be half an hour late, we make a new booking from the old one with copy, naming only the field that differs, and the original still says 19:30. Java’s records have no equivalent, even in Java 25: to change one component of a record, you call the constructor again and pass every component. Making a changed copy and leaving the original alone is the functional-style habit I described in chapter 1.

3.2. Methods on a case class

Chapter 2’s rules were top-level methods that took a party size. A case class can have methods of its own. That lets the rules move onto Party and gives Booking a way to work out when its table has to be given back. Replace Party and Booking with these versions:

case class Party(name: String, size: Int):
  def stayMinutes: Int =
    if size <= 4 then 90 else 120

  def deposit: BigDecimal =
    if size >= 6 then BigDecimal("10.00") * size else BigDecimal(0)

case class Booking(party: Party, table: Table, time: LocalTime):
  def leaveBy: LocalTime = time.plusMinutes(party.stayMinutes)

The colon at the end of case class Party(name: String, size: Int): opens the body of the class, much as = opens the body of a method. Inside it, size is the party’s own field. Since none of the three methods takes arguments and each just returns a value, the convention from chapter 1 says to declare them without parentheses. Calling one looks the same as reading a field. The REPL only reads model.scala when it starts. Quit it and start it again after every change to the file:

scala> import java.time.LocalTime

scala> val booking = Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30))
val booking: Booking = Booking(
  party = Party(name = "Okonkwo", size = 5),
  table = Table(number = 5, seats = 6),
  time = 19:30
)

scala> booking.leaveBy
val res0: LocalTime = 21:30

scala> booking.party.deposit
val res1: BigDecimal = 0

scala> Party("Lindqvist", 7).deposit
val res2: BigDecimal = 70.00

3.3. Statuses and enums

A booking moves through stages: somebody requests it, the restaurant confirms it, the party turns up and is seated, and at the end of the evening it’s complete. It can also be cancelled, or the party can fail to show. A fixed set of named values like that is an enum. Add an enum above Booking, and replace Booking with a version that has a status:

enum Status:
  case Requested, Confirmed, Seated, Completed, Cancelled, NoShow

case class Booking(
    party: Party,
    table: Table,
    time: LocalTime,
    status: Status = Status.Requested
):
  def leaveBy: LocalTime = time.plusMinutes(party.stayMinutes)

Each value of the enum is introduced by case. One case can introduce several values. The fourth field of Booking has a default value. The three-argument bookings we’ve written so far still compile, and they start out as Requested. With more than two or three parameters, one to a line is easier to read, so that’s how I’ve laid them out.

scala> import java.time.LocalTime

scala> Status.values
val res0: Array[Status] = Array(Requested, Confirmed, Seated, Completed, Cancelled, NoShow)

scala> Status.valueOf("Seated")
val res1: Status = Seated

scala> val booking = Booking(Party("Patel", 2), Table(1, 2), LocalTime.of(19, 30))
val booking: Booking = Booking(
  party = Party(name = "Patel", size = 2),
  table = Table(number = 1, seats = 2),
  time = 19:30,
  status = Requested
)

scala> booking.copy(status = Status.Confirmed)
val res2: Booking = Booking(
  party = Party(name = "Patel", size = 2),
  table = Table(number = 1, seats = 2),
  time = 19:30,
  status = Confirmed
)

values and valueOf behave as they do on a Java enum. The Array that values returns is a Java array. Scala’s enums go further than Java’s, because each case can have fields that the other cases don’t. Chapter 8 uses that.

3.4. Choosing with match

The staff need to see a status in words. Add this to the end of model.scala:

def describe(status: Status): String = status match
  case Status.Requested => "waiting to be confirmed"
  case Status.Confirmed => "confirmed"
  case Status.Seated    => "at the table"
  case Status.Completed => "been and gone"
  case Status.Cancelled => "cancelled"
  case Status.NoShow    => "never turned up"

A match looks like a Java switch. On an enum it behaves like a Java switch expression written with arrows, in that the first case that fits is chosen and there’s no falling through to the next one. A match is an expression, as if is, so describe can return it directly.

Knowing every value of Status, the compiler can tell when a match has left one out. Delete the NoShow line and compile:

$ scala compile model.scala
[warn] ./model.scala:26:40
[warn] match may not be exhaustive.
[warn]
[warn] It would fail on pattern case: NoShow
[warn] def describe(status: Status): String = status match
[warn]                                        ^^^^^^

It’s only a warning, so the file still compiles, although calling describe with NoShow would throw a MatchError at run time. I tend to make all of my warnings hard errors, which we set up in chapter 18. Put the line back.

The check is most useful when the code changes. Add a seventh status next year and the compiler will point out each match like this one that doesn’t handle it yet.

Several values can share a case. Add this method as well:

def holdsTable(booking: Booking): Boolean = booking.status match
  case Status.Confirmed | Status.Seated => true
  case _                                => false

| between patterns means “or”. _ is a pattern that matches anything. On the last line it works as a catch-all, like Java’s default. Because it covers every value, a catch-all also silences the missing-case warning for that match. Save it for when “everything else” is what you mean.

Patterns can also take a case class apart. Add this third method:

def label(booking: Booking): String = booking match
  case Booking(Party(name, size), Table(number, _), time, Status.Confirmed) if size >= 6 =>
    s"$time table $number: $name, party of $size (deposit taken)"
  case Booking(Party(name, size), Table(number, _), time, _) =>
    s"$time table $number: $name, party of $size"

The first pattern, Booking(Party(name, size), Table(number, _), time, Status.Confirmed), matches only a confirmed booking. In the same step it gives names to the parts we want to use: name, size, number and time. The _ inside Table(number, _) skips the seat count. The if that follows is a guard, a further condition the case has to pass. Because cases are tried from the top, the more specific one has to come first.

scala> import java.time.LocalTime

scala> describe(Status.NoShow)
val res0: String = "never turned up"

scala> val patel = Booking(Party("Patel", 2), Table(1, 2), LocalTime.of(19, 30))
val patel: Booking = Booking(
  party = Party(name = "Patel", size = 2),
  table = Table(number = 1, seats = 2),
  time = 19:30,
  status = Requested
)

scala> holdsTable(patel)
val res1: Boolean = false

scala> label(patel)
val res2: String = "19:30 table 1: Patel, party of 2"

scala> label(Booking(Party("Lindqvist", 7), Table(6, 8), LocalTime.of(20, 0), Status.Confirmed))
val res3: String = "20:00 table 6: Lindqvist, party of 7 (deposit taken)"

Java can do a fair amount of this now. Its switch has had patterns for records since Java 21, and a switch expression over an enum has always had to cover every constant. Scala has had match since its early versions, and you’ll probably meet it far more often than you meet switch in Java.

3.5. Where statics went

Scala has no static keyword. A value or method that doesn’t belong to any one instance goes in an object, which defines a class and its single instance in one go. When an object shares its name with a class in the same file, it’s called that class’s companion. It’s the usual home for what Java would make static members of the class. Add one for Booking, again at the end of the file:

object Booking:
  val largestParty = 8

  def walkIn(party: Party, table: Table, now: LocalTime): Booking =
    Booking(party, table, now, Status.Seated)
scala> import java.time.LocalTime

scala> Booking.largestParty
val res0: Int = 8

scala> Booking.walkIn(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15))
val res1: Booking = Booking(
  party = Party(name = "Moreau", size = 3),
  table = Table(number = 3, seats = 4),
  time = 20:15,
  status = Seated
)

Booking.walkIn is called like a static factory method. A real Java static such as LocalTime.of looks the same from Scala. The compiler gives every case class a companion whether you write one or not. It contains a method that builds instances, which is what runs when you write Table(5, 6) with no new.

describe and label are top-level methods that could have gone in a companion. I put a method in a companion when it’s about that one type and leave general helpers at the top level.

3.6. Classes and traits

A case class suits data that doesn’t change. The collection of bookings for an evening changes as bookings arrive. For that we want an ordinary class, which I’ll put behind a trait. Add both to model.scala:

trait BookingStore:
  def add(booking: Booking): Unit
  def all: List[Booking]

  def at(time: LocalTime): List[Booking] =
    all.filter(booking => booking.time == time)

class InMemoryBookingStore extends BookingStore:
  private var bookings: List[Booking] = List()

  def add(booking: Booking): Unit =
    bookings = bookings.appended(booking)

  def all: List[Booking] = bookings

A trait is Scala’s version of an interface. BookingStore declares two methods with no body and gives a third, at, a body written in terms of all, as a Java interface can with a default method. InMemoryBookingStore is a plain class that extends the trait and supplies the missing two. A class can extend several traits, separated by commas.

The class keeps its bookings in a private var, the first mutable state in Fully Booked. Only add can change it, by building a new list with appended and storing that in the var. Since an empty List() gives the compiler no element type to infer, the type of bookings is written out.

A plain class doesn’t get the generated methods a case class has. It has no toString, equals or copy of its own, and two stores are equal only when they’re the same object. That suits something with identity and changing state. A class can take parameters as a case class does, class InMemoryBookingStore(limit: Int) for instance, but they’re private to the class unless you mark them val. Scala 3 lets you create an instance of any class without new, as in InMemoryBookingStore().

The trait is there because a second implementation is coming. When the bookings move into a database in chapter 20, code written against BookingStore won’t need to change.

3.7. One project, several files

With data types, display code and storage all in model.scala, it’s time to split the file up.

Create a file called project.scala for the directives, so that they’re stated once for the whole directory:

//> using scala 3.9.0
//> using jvm 25

model.scala keeps the case classes, the enum and the Booking companion. describe, holdsTable and label move to desk.scala, and the trait and its class to store.scala. Take the directives out of model.scala. Each of those three files then starts with a package declaration, followed by the LocalTime import in model.scala and store.scala, which use it:

package fullybooked

import java.time.LocalTime

Packages work as they do in Java, with the exception that the directory structure doesn’t have to match the package name. All the files can sit side by side. Code in one file can use anything from the others in the same package without an import, including top-level methods like label.

A fifth file, main.scala, belongs to the same package and puts the pieces to work:

package fullybooked

import java.time.LocalTime

@main def frontDesk(): Unit =
  val store = InMemoryBookingStore()
  store.add(Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30), Status.Confirmed))
  store.add(Booking(Party("Patel", 2), Table(1, 2), LocalTime.of(19, 30)))
  store.add(Booking(Party("Lindqvist", 7), Table(6, 8), LocalTime.of(20, 0), Status.Confirmed))
  store.add(Booking.walkIn(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15)))

  for booking <- store.all do
    println(s"${label(booking)}, ${describe(booking.status)}")

Run the whole directory by giving the scala command a . in place of a file name:

$ scala run .
19:30 table 5: Okonkwo, party of 5, confirmed
19:30 table 1: Patel, party of 2, waiting to be confirmed
20:00 table 6: Lindqvist, party of 7 (deposit taken), confirmed
20:15 table 3: Moreau, party of 3, at the table

The for from chapter 2 walks through a list the same way it walked through a range. store.all is a List[Booking]. Chapter 4 is about what you can do with one.

3.8. Exercises

Solutions are in the book’s downloadable code, under ch03/exercises.

  1. Give Table a method fits(party: Party): Boolean. Try it in the REPL: scala repl . loads the whole project, and import fullybooked.* brings its contents into scope.

  2. Add a Waitlisted case to Status and compile the project with scala compile .. Which match does the compiler warn you about, and why does it say nothing about holdsTable?

  3. Write canSeat(booking: Booking): Boolean. Only a confirmed booking can be shown to its table.

  4. Give Table a companion object holding the restaurant’s six tables as Table.all, then use fits to print the tables that would take a party of 7.

  5. Add a method count: Int to BookingStore, with a body, without touching InMemoryBookingStore. Call it from main.scala.

4. Collections

Over an evening the staff need answers from the bookings, such as how many people are coming and whether anything is free at eight. Each of those is a question about a list. In Scala you answer it with methods on the list itself, where Java would use a loop or a stream. I said in chapter 1 that this is where the differences from Java start to matter, so most of this chapter is spent at the REPL, trying those methods on real bookings.

4.1. An evening’s bookings

Add a file called tonight.scala to the project:

package fullybooked

import java.time.LocalTime

val tonight = List(
  Booking(Party("Haddad", 6), Table(5, 6), LocalTime.of(18, 0), Status.Cancelled),
  Booking(Party("Nakamura", 2), Table(2, 2), LocalTime.of(18, 30), Status.Seated),
  Booking(Party("Abara", 4), Table(4, 4), LocalTime.of(18, 45), Status.Seated),
  Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30), Status.Confirmed),
  Booking(Party("Patel", 2), Table(1, 2), LocalTime.of(19, 30)),
  Booking(Party("Lindqvist", 7), Table(6, 8), LocalTime.of(20, 0), Status.Confirmed),
  Booking(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15), Status.Confirmed),
  Booking(Party("Fitzgerald", 2), Table(2, 2), LocalTime.of(21, 15), Status.Confirmed)
)

We’ll also need the restaurant’s tables. Put them in a companion for Table, in model.scala. If you did exercise 4 in chapter 3, you’ll have this already:

object Table:
  val all = List(
    Table(1, 2), Table(2, 2), Table(3, 4),
    Table(4, 4), Table(5, 6), Table(6, 8)
  )

Start the REPL on the whole project by running scala repl . and import two things before anything else, the project’s package and LocalTime:

scala> import fullybooked.*

scala> import java.time.LocalTime

scala> tonight.size
val res0: Int = 8

scala> tonight.map(booking => booking.party.name)
val res1: List[String] = List(
  "Haddad",
  "Nakamura",
  "Abara",
  "Okonkwo",
  "Patel",
  "Lindqvist",
  "Moreau",
  "Fitzgerald"
)

scala> tonight.filter(booking => booking.party.size >= 6).map(booking => booking.party.name)
val res2: List[String] = List("Haddad", "Lindqvist")

map calls the lambda on every element and returns a list of the results, so a list of bookings becomes a list of names. Chaining it after filter, which we used in chapter 1, reads left to right: keep the big parties, then take their names. The Java for that last line would be tonight.stream().filter(...).map(...).toList(), with the same two lambdas in the middle.

I’ll leave the two imports out of the sessions that follow. My sessions each number their results from res0, while yours will carry on counting.

4.2. Shorter lambdas

A lambda like booking => booking.party.name gives its argument a name only to use it once. For that case Scala has a shorthand, _.party.name, in which the underscore stands for the argument:

scala> tonight.map(_.party.size).sum
val res0: Int = 31

scala> tonight.count(holdsTable)
val res1: Int = 6

scala> tonight.sortBy(_.party.size).map(_.party.name).take(3)
val res2: List[String] = List("Nakamura", "Patel", "Fitzgerald")

scala> tonight.maxBy(_.party.size).party.name
val res3: String = "Lindqvist"

scala> tonight.exists(_.status == Status.Cancelled)
val res4: Boolean = true

scala> tonight.forall(_.party.size <= Booking.largestParty)
val res5: Boolean = true

Tonight’s eight bookings add up to 31 people, the cancelled party included. Six of the eight currently hold a table. A method’s name can go wherever a lambda is expected. That’s how count was handed holdsTable directly. The rest will look familiar if you’ve used Java streams: sortBy orders the list by whatever the lambda returns, take(3) keeps the first three, and exists and forall are anyMatch and allMatch under other names.

The underscore only works when the argument is used once. I use it for short things like _.party.size and go back to a named argument as soon as the lambda does any real work.

sortBy also works with Java types that implement Comparable, such as LocalTime. Change frontDesk in main.scala so that it prints tonight’s bookings in order of arrival, and run it from a second terminal so that the REPL stays open:

@main def frontDesk(): Unit =
  for booking <- tonight.sortBy(_.time) do
    println(s"${label(booking)}, ${describe(booking.status)}")
$ scala run . --main-class fullybooked.frontDesk
18:00 table 5: Haddad, party of 6, cancelled
18:30 table 2: Nakamura, party of 2, at the table
18:45 table 4: Abara, party of 4, at the table
19:30 table 5: Okonkwo, party of 5, confirmed
19:30 table 1: Patel, party of 2, waiting to be confirmed
20:00 table 6: Lindqvist, party of 7 (deposit taken), confirmed
20:15 table 3: Moreau, party of 3, confirmed
21:15 table 2: Fitzgerald, party of 2, confirmed

The --main-class option is there because this chapter adds a second @main to the project later on. With more than one, the scala command has to be told which to run, by its full name including the package.

4.3. When there might be no answer

Look for a booking by the party’s name:

scala> tonight.find(_.party.name == "Patel")
val res0: Option[Booking] = Some(
  Booking(
    party = Party(name = "Patel", size = 2),
    table = Table(number = 1, seats = 2),
    time = 19:30,
    status = Requested
  )
)

scala> tonight.find(_.party.name == "Smith")
val res1: Option[Booking] = None

The Patel booking comes back wrapped in Some. The search for Smith finds no booking and returns None. Both results have the type Option[Booking]. A Scala method uses Option in its return type to say that there might be no answer. It’s also the fix for the loose end in chapter 1, where minBy had to be protected by an isEmpty check that was easy to leave out. Chapter 5 is about Option. Until then, be aware that minBy and maxBy assume there’s an answer and throw an exception when there isn’t.

4.4. Grouping and maps

How many people is the kitchen cooking for in each hour?

scala> val covers = tonight.filter(holdsTable).groupBy(_.time.getHour).map((hour, bookings) => (hour, bookings.map(_.party.size).sum))
val covers: Map[Int, Int] = Map(20 -> 10, 21 -> 2, 18 -> 6, 19 -> 5)

scala> covers(19)
val res0: Int = 5

scala> covers.getOrElse(22, 0)
val res1: Int = 0

scala> covers.toList.sorted
val res2: List[(Int, Int)] = List((18, 6), (19, 5), (20, 10), (21, 2))

Read the long line from the left. Once filter has dropped the two bookings that don’t hold a table, groupBy puts the rest into groups by whatever its lambda returns, which here is the hour of arrival, and hands back a Map from each hour to the list of bookings in it. A list per hour is more than we asked for. The final map goes through the entries of that Map, each of them a key with its value. A lambda with two arguments, such as (hour, bookings) => ..., receives the two separately. Ours gives back the hour with a head count, making the final result a Map from hours to people.

The REPL prints each entry as 20 -> 10. The arrow is also how you write an entry yourself, as in Map(18 -> 6, 19 -> 5). 18 -> 6 is another way of writing the tuple (18, 6).

Notice the order the entries came back in: 20, 21, 18, 19. A Map doesn’t keep any order. To print one sensibly, turn it into a list of pairs and sort that, as the last line does. A lookup is written like a method call, covers(19). A plain lookup of a missing key throws an exception, so if the key might not be there, use getOrElse and say what you want back.

4.5. Which tables are free

scala> val taken = tonight.filter(holdsTable).map(_.table.number).toSet
val taken: Set[Int] = Set(5, 6, 2, 3, 4)

scala> taken.contains(1)
val res0: Boolean = false

scala> Table.all.map(_.number).filterNot(taken.contains)
val res1: List[Int] = List(1)

Two of the bookings that hold a table tonight are for table 2, but a Set holds a value only once, so 2 appears once in taken. Like a Map, a Set has no order. What a Set is good at is answering contains quickly, however big it gets. Handing taken.contains to filterNot leaves the tables that aren’t in the set. Tonight that’s only table 1.

That answer ignores the time, though. A table is really taken at a given moment only if a booking that holds it has started and hasn’t finished. Add a new file, availability.scala:

package fullybooked

import java.time.LocalTime

def freeAt(time: LocalTime, bookings: List[Booking]): List[Table] =
  val taken = bookings
    .filter(holdsTable)
    .filter(booking => !time.isBefore(booking.time) && time.isBefore(booking.leaveBy))
    .map(_.table)
    .toSet
  Table.all.filterNot(taken.contains)

The second filter keeps a booking if time isn’t before its start and is before the leaveBy we gave Booking in chapter 3. Restart the REPL, repeat the two imports, and ask about two different times:

scala> freeAt(LocalTime.of(20, 0), tonight).map(_.number)
val res0: List[Int] = List(1, 2, 3)

scala> freeAt(LocalTime.of(18, 45), tonight).map(_.number)
val res1: List[Int] = List(1, 3, 5, 6)

freeAt is still a simplification, because it only asks about one moment. Table 3 is offered at eight although the Moreaus have it from quarter past. Exercise 7 fixes that.

4.6. Flattening and folding

Two more methods are worth knowing, for the cases the others don’t cover. The first is for a lambda that returns a list of its own. Here the lambda gives the arrival and leaving times of a booking, and we apply it to the first three. The session’s last line is the second method, foldLeft:

scala> tonight.take(3).map(booking => List(booking.time, booking.leaveBy))
val res0: List[List[LocalTime]] = List(List(18:00, 20:00), List(18:30, 20:00), List(18:45, 20:15))

scala> tonight.take(3).flatMap(booking => List(booking.time, booking.leaveBy))
val res1: List[LocalTime] = List(18:00, 20:00, 18:30, 20:00, 18:45, 20:15)

scala> tonight.foldLeft(0)((total, booking) => total + booking.party.size)
val res2: Int = 31

map produces a list of lists, one inner list for each booking. flatMap does the same work and then joins the inner lists end to end, leaving a single list of times. Use it whenever each element turns into several.

foldLeft is for a calculation that no existing method does. It takes its arguments in two separate pairs of parentheses, a peculiarity that chapter 7 explains. The first pair holds a starting value. The second holds a lambda with two arguments, the result so far and the next element. foldLeft calls it for each element in turn, carrying the result along. The foldLeft above starts at 0 and adds each party’s size, arriving at the same 31 that map and sum gave earlier. Before writing a fold, check that the method you want doesn’t already exist: sum, count and maxBy could all be written as folds.

4.7. for with yield

The for loops so far have ended in do and run their body for what it does (its “side effects”), such as printing. Ending one in yield makes it build a collection:

scala> for booking <- tonight if booking.party.size >= 6 yield booking.party.name
val res0: List[String] = List("Haddad", "Lindqvist")

scala> for table <- Table.all.take(2); hour <- List(18, 20) yield (table.number, hour)
val res1: List[(Int, Int)] = List((1, 18), (1, 20), (2, 18), (2, 20))

The first has an if after its <- clause, which makes it filter followed by map, written another way. The second has two <- clauses separated by a semicolon and produces every combination of a table and an hour, like a nested loop. Chapter 10 explains how the compiler turns both into calls to the collection methods. I use method chains when there’s one collection involved and for when several are being combined, because that’s where a chain of nested lambdas gets hard to read.

4.8. Lists that don’t change

No method so far has changed the collection it was called on. Adding an element doesn’t change it either:

scala> val seats = Table.all.map(_.seats)
val seats: List[Int] = List(2, 2, 4, 4, 6, 8)

scala> val more = 10 :: seats
val more: List[Int] = List(10, 2, 2, 4, 4, 6, 8)

scala> seats
val res0: List[Int] = List(2, 2, 4, 4, 6, 8)

scala> more.tail eq seats
val res1: Boolean = true

scala> seats :+ 10
val res2: List[Int] = List(2, 2, 4, 4, 6, 8, 10)

scala> seats ++ List(10, 12)
val res3: List[Int] = List(2, 2, 4, 4, 6, 8, 10, 12)

scala> seats(4)
val res4: Int = 6

In chapter 2 I said I’d tell you how to read operators like these. :: is read “cons” and puts an element on the front of a list. :+ is read “append” and is the appended method from chapter 3 under another name. ++ is “concat” and joins two collections. None of them changed seats. It still has six elements after more was built from it.

Copying a whole list every time you add to it sounds slow, and more.tail eq seats shows why it isn’t. tail is a list without its first element, and eq is the reference comparison from chapter 2. The answer is true: the tail of more is the very same object as seats. Because a List is a linked list, putting 10 on the front only made one new cell, which points at the existing list. That sharing is only safe because neither list can ever change.

Working at the other end of a List is slower. :+ has to copy the whole list, and seats(4) has to walk along it to reach the fifth element. For a few dozen bookings that doesn’t matter. For a long sequence that you append to or index into, use Vector, which works like List apart from :: and does both quickly. You’ll mostly see List in Scala code, with Vector where it’s needed and Seq, a type that covers both, in method signatures that don’t mind which they get.

4.9. Mutable and Java collections

Scala has mutable collections as well, in the package scala.collection.mutable. Converting between Scala’s collections and Java’s, in either direction, takes one import:

scala> import scala.jdk.CollectionConverters.*

scala> val fromJava = java.util.List.of(2, 4, 6).asScala.toList
val fromJava: List[Int] = List(2, 4, 6)

scala> fromJava.asJava
val res0: java.util.List[Int] = [2, 4, 6]

scala> val buffer = scala.collection.mutable.ArrayBuffer(2, 4)
val buffer: collection.mutable.ArrayBuffer[Int] = ArrayBuffer(2, 4)

scala> buffer += 6
val res1: collection.mutable.ArrayBuffer[Int] = ArrayBuffer(2, 4, 6)

scala> buffer.toList
val res2: List[Int] = List(2, 4, 6)

Since asScala and asJava wrap the collection they’re given instead of copying it, they’re quick to call at the point where your code meets a Java library. What asScala returns is a view of the Java list, which can still change underneath you. That’s why the example copies it into a List with toList straight away. ArrayBuffer is the counterpart of Java’s ArrayList. Its += adds an element in place. A mutable collection is a reasonable choice inside one method that’s building something up step by step. Convert it with toList before handing the result to anyone else.

4.10. The last table at eight

Chapter 1 mentioned two people asking for the last table at eight o’clock at the same moment. We now have enough code to get that wrong. Add a method to availability.scala that makes a booking:

def reserve(store: BookingStore, party: Party, time: LocalTime): String =
  val candidates = freeAt(time, store.all).filter(_.seats >= party.size)
  if candidates.isEmpty then s"${party.name}: sorry, nothing free at $time"
  else
    val table = candidates.minBy(_.seats)
    Thread.sleep(300)
    store.add(Booking(party, table, time, Status.Confirmed))
    s"${party.name}: table ${table.number} at $time"

reserve picks the smallest free table that’s big enough and adds a confirmed booking to the store. The Thread.sleep represents the time a real system spends between checking and saving, on a call to a database for instance. I’ve made it 300 milliseconds, far longer than is realistic, so that the program below does the same thing on every run.

Then add a second @main to main.scala:

@main def lastTable(): Unit =
  val eight = LocalTime.of(20, 0)
  val store = InMemoryBookingStore()
  for table <- Table.all.filter(_.number != 6) do
    store.add(Booking(Party("Regular", table.seats), table, eight, Status.Seated))
  println(s"Free at $eight: ${freeAt(eight, store.all).map(_.number)}")

  var toIbrahim = ""
  var toCastellanos = ""
  val first = Thread(() => toIbrahim = reserve(store, Party("Ibrahim", 7), eight))
  val second = Thread(() => toCastellanos = reserve(store, Party("Castellanos", 8), eight))
  first.start()
  Thread.sleep(100)
  second.start()
  first.join()
  second.join()

  println(toIbrahim)
  println(toCastellanos)
  val onTableSix = store.at(eight).filter(_.table.number == 6).map(_.party.name)
  println(s"Table 6 at $eight: $onTableSix")

The program fills every table except number 6 for eight o’clock. Then two parties who both need table 6 try to book it from separate Java threads, the second starting a tenth of a second after the first. A Scala lambda can be passed where a Runnable is wanted, as a Java lambda can. Each thread’s answer goes into a var so that both can be printed in a fixed order at the end. Scala lets a lambda assign to a local var, where Java would insist on an effectively final variable.

$ scala run . --main-class fullybooked.lastTable
Free at 20:00: List(6)
Ibrahim: table 6 at 20:00
Castellanos: table 6 at 20:00
Table 6 at 20:00: List(Ibrahim, Castellanos)

Both parties have been told they’ve got table 6, and the store holds both bookings. Each thread looked before the other had saved, so each saw table 6 free and went ahead. Java code written the same way would have the same bug. Immutable lists don’t prevent it: the list in the store never changes, but the var that points at it does change, and the check and the booking are two separate steps.

add has a second bug of the same kind. Two threads can each read the same list, append to it and store their own result, in which case the later one overwrites the earlier. While I was trying different timings for this example, some runs ended with only one of the two bookings in the store.

The sleeps only fix the timing, so that the double booking shows on every run, and both bugs would be there without them. In a real system the gap between checking and saving is much shorter. The double booking then needs two requests to arrive inside that gap, which on a busy night they eventually will. This problem comes back in chapters 14, 17 and 20, each of which fixes it in a different way.

4.11. Exercises

Solutions are in the book’s downloadable code, under ch04/exercises.

  1. Work out the total deposit due tonight from the parties that hold a table. sum works on a list of BigDecimal.

  2. List the names of the parties arriving at eight or later, in order of arrival.

  3. Count how many bookings there are in each status. The result should be a Map.

  4. Which tables are booked more than once tonight? filter on a Map takes a two-argument lambda as map does, and keys gives you the keys of what’s left.

  5. Predict the results of List[Int]().forall(_ > 5) and List[Int]().exists(_ > 5), then try them. List[Int]() is an empty list of integers.

  6. Take the Thread.sleep(300) out of reserve and run lastTable several times. Then take the Thread.sleep(100) out of lastTable as well and run it several more. What changes each time, and why?

  7. Write freeFor(party: Party, time: LocalTime, bookings: List[Booking]): List[Table], which returns the tables that are big enough and stay free for the party’s whole visit. Two time ranges overlap when each starts before the other ends.

5. Option, and life without null

Someone rings the restaurant to say they’re running late and gives a surname. When the staff search for the booking, there might not be one under that name. Java code might return null from that search, or an Optional if its author was being careful. In Scala the search returns an Option. A field or a parameter whose value might be missing is an Option as well, so you’ll hardly ever see null written in Scala.

5.1. Looking up a booking

Add a method to the end of desk.scala that searches a list of bookings by name:

def findBooking(name: String, bookings: List[Booking]): Option[Booking] =
  bookings.find(_.party.name == name)

A list’s find returns an Option[Booking], a value that holds a booking or nothing. findBooking passes it straight on. To try it from the command line, add a third @main to the end of main.scala, one that takes the name as its parameter:

@main def lookUp(name: String): Unit =
  val booking = findBooking(name, tonight)
  println(label(booking))
$ scala compile .
[error] ./main.scala:33:17
[error] Found:    (booking : Option[fullybooked.Booking])
[error] Required: fullybooked.Booking
[error]   println(label(booking))
[error]                 ^^^^^^^
Compilation failed

We’ve given label an Option[Booking] where it wants a Booking. Had findBooking returned a Booking that could be null, this version would have compiled, printed the Patel booking and crashed on the null it got for Smith. With an Option, the compiler won’t let us reach the booking until we’ve said what happens when there isn’t one. Replace lookUp with a version that handles both cases:

@main def lookUp(name: String): Unit =
  findBooking(name, tonight) match
    case Some(booking) => println(label(booking))
    case None          => println(s"No booking under $name")

An Option is either a Some holding a value or None. Some is a case class with one field, so Some(booking) is a pattern like the ones that took a Booking apart in chapter 3. It matches when there’s a value and gives that value the name booking. The None case matches when there’s no value. Leave either case out and the compiler warns that the match is missing one.

$ scala run . --main-class fullybooked.lookUp -- Patel
19:30 table 1: Patel, party of 2
$ scala run . --main-class fullybooked.lookUp -- Smith
No booking under Smith

An Option also has a get method which, like Optional.get in Java, returns the value or throws a NoSuchElementException when there’s none. Calling get on the Smith search would crash lookUp as a null would have. I don’t use get and I ask for it to be changed when I see it in a review.

5.2. A field that might be empty

The staff would like to ring a party that’s late, but not every party leaves a number. In Java this is often a field that may be null: the documentation for Optional says it’s mainly intended for return types. Scala code uses Option for a field that might be empty as well. Change the first line of Party in model.scala:

case class Party(name: String, size: Int, phone: Option[String] = None):

The default means every Party("Nakamura", 2) we’ve already written still compiles. The two patterns in label don’t, because they list the fields of Party by position. With a third field added, each pattern needs a _ to skip it:

def label(booking: Booking): String = booking match
  case Booking(Party(name, size, _), Table(number, _), time, Status.Confirmed) if size >= 6 =>
    s"$time table $number: $name, party of $size (deposit taken)"
  case Booking(Party(name, size, _), Table(number, _), time, _) =>
    s"$time table $number: $name, party of $size"

Whenever you add a field to a case class, the project stops compiling at every pattern that lists that class’s fields.

A party with a number gets it wrapped in Some. In tonight.scala, replace the four bookings from Okonkwo to Moreau with these:

  Booking(Party("Okonkwo", 5, Some("07700 900141")), Table(5, 6),
    LocalTime.of(19, 30), Status.Confirmed),
  Booking(Party("Patel", 2, Some("07700 900129")), Table(1, 2),
    LocalTime.of(19, 30)),
  Booking(Party("Lindqvist", 7, Some("07700 900152")), Table(6, 8),
    LocalTime.of(20, 0), Status.Confirmed),
  Booking(Party("Moreau", 3, Some("07700 900164")), Table(3, 4),
    LocalTime.of(20, 15), Status.Confirmed),

Start the REPL with scala repl ., import fullybooked.* and java.time.LocalTime again, and ask about the Moreau party:

scala> findBooking("Moreau", tonight).map(_.party.size)
val res0: Option[Int] = Some(3)

scala> findBooking("Moreau", tonight).map(_.party.phone)
val res1: Option[Option[String]] = Some(Some("07700 900164"))

scala> findBooking("Moreau", tonight).flatMap(_.party.phone)
val res2: Option[String] = Some("07700 900164")

scala> findBooking("Abara", tonight).flatMap(_.party.phone)
val res3: Option[String] = None

scala> findBooking("Smith", tonight).flatMap(_.party.phone)
val res4: Option[String] = None

On an Option, map applies the lambda to the value if there is one and gives back the result in a Some. It leaves a None as it is. In the second line the lambda returns an Option of its own, the phone number, so the result is an Option inside an Option. flatMap removes the extra layer, much as it joined inner lists into one list. Use it whenever the next step might also come back empty.

The last two lines both give None: the Abaras have a booking and left no number, while nobody called Smith has a booking at all.

Replace lookUp once more, so that it prints the number:

@main def lookUp(name: String): Unit =
  findBooking(name, tonight) match
    case Some(booking) =>
      val phone = booking.party.phone.getOrElse("none given")
      println(label(booking))
      println(s"Phone: $phone")
    case None =>
      println(s"No booking under $name")

Here getOrElse returns the value if there is one and its argument if there isn’t.

$ scala run . --main-class fullybooked.lookUp -- Moreau
20:15 table 3: Moreau, party of 3
Phone: 07700 900164
$ scala run . --main-class fullybooked.lookUp -- Abara
18:45 table 4: Abara, party of 4
Phone: none given

5.3. Options and lists

An Option behaves like a collection that holds one value or none. It has most of the methods a list has and can be mixed with lists:

scala> tonight.map(_.party.phone)
val res0: List[Option[String]] = List(
  None,
  None,
  None,
  Some("07700 900141"),
  Some("07700 900129"),
  Some("07700 900152"),
  Some("07700 900164"),
  None
)

scala> tonight.flatMap(_.party.phone)
val res1: List[String] = List("07700 900141", "07700 900129", "07700 900152", "07700 900164")

scala> findBooking("Moreau", tonight).exists(_.party.size > 2)
val res2: Boolean = true

scala> findBooking("Haddad", tonight).filter(holdsTable)
val res3: Option[Booking] = None

Mapping tonight’s bookings to their phone numbers gives a list of options. flatMap treats each Option as a list of one or nothing and joins them up, which leaves the four numbers we have. That makes flatMap the method for a list where only some of the elements have the value you want. On a single Option, exists asks whether there’s a value that passes the test, and filter turns a Some into a None when its value fails. The Haddads have a booking, but it’s cancelled and doesn’t hold a table.

I don’t wrap a List in an Option, because an empty list already says there’s nothing. With Option[List[Booking]], callers would have two kinds of nothing to check for.

Sometimes two parties ask to swap tables. The staff can only do it if both bookings are found. Add this to the end of desk.scala:

def swapTables(
    first: String,
    second: String,
    bookings: List[Booking]
): Option[(Booking, Booking)] =
  for
    a <- findBooking(first, bookings)
    b <- findBooking(second, bookings)
  yield (a.copy(table = b.table), b.copy(table = a.table))

A for with yield works on options too. Its clauses usually go on separate lines beneath the for, as here, and then they need no semicolon between them. Each <- takes the booking out of an Option. When either search comes back as None, the yield doesn’t run and the whole result is None. Otherwise the result is a Some holding a tuple of two copies, each with the other’s table. Restart the REPL to pick up the new method and repeat the two imports:

scala> swapTables("Moreau", "Abara", tonight).map((a, b) => (a.table.number, b.table.number))
val res0: Option[(Int, Int)] = Some((4, 3))

scala> swapTables("Moreau", "Smith", tonight)
val res1: Option[(Booking, Booking)] = None

5.4. Asking for one element

Our reserve still has a minBy that throws on an empty list and an isEmpty check to protect it. The library has a second version of nearly every method that can throw like that, one that returns an Option:

scala> "19".toIntOption
val res0: Option[Int] = Some(19)

scala> "seven".toIntOption
val res1: Option[Int] = None

scala> tonight.headOption.map(_.party.name)
val res2: Option[String] = Some("Haddad")

scala> List[Booking]().headOption
val res3: Option[Booking] = None

scala> val covers = Map(18 -> 6, 19 -> 5)
val covers: Map[Int, Int] = Map(18 -> 6, 19 -> 5)

scala> covers.get(19)
val res4: Option[Int] = Some(5)

scala> covers.get(22)
val res5: Option[Int] = None

The versions that throw are toInt on the string, head on the list and the plain covers(22) lookup. The names are mostly regular: head, last, max, min, maxBy and minBy all gain an Option version by adding the word to the end. So do the string conversions, where toIntOption saves catching the NumberFormatException that Java’s Integer.parseInt throws. Map is the odd one out, with a plain get.

With minByOption, reserve can lose its isEmpty check. Add a method to availability.scala, above reserve, that picks a table without booking it:

def bestTable(party: Party, time: LocalTime, bookings: List[Booking]): Option[Table] =
  freeAt(time, bookings)
    .filter(_.seats >= party.size)
    .minByOption(_.seats)

If nothing is free, or nothing free is big enough, the answer is None. Now replace reserve with a version built on bestTable:

def reserve(store: BookingStore, party: Party, time: LocalTime): Option[Booking] =
  bestTable(party, time, store.all) match
    case None => None
    case Some(table) =>
      Thread.sleep(300)
      val booking = Booking(party, table, time, Status.Confirmed)
      store.add(booking)
      Some(booking)

This reserve returns the booking it made, if it made one, and leaves the wording to its caller. The old version returned a String for the customer. A match that turns None into None and a Some into another Some is map written out by hand. I’ve kept the match because the second case sleeps and saves before it builds its value, which takes several lines. Chapter 7 shows how to write a lambda that long.

The project won’t compile yet, because lastTable still expects a String from reserve. First add the wording to the end of desk.scala:

def reply(name: String, result: Option[Booking]): String =
  result
    .map(booking => s"$name: table ${booking.table.number} at ${booking.time}")
    .getOrElse(s"$name: sorry, nothing free")

In reply, map changes the value if it’s there and getOrElse says what to use if it isn’t. A match with two cases would do the same job. So would fold, which takes the fallback and the lambda in one call. I find map and getOrElse easier to read than fold.

Then, in lastTable, replace everything from var toIbrahim down to the two printlns that follow the joins:

  var toIbrahim: Option[Booking] = None
  var toCastellanos: Option[Booking] = None
  val first = Thread(() => toIbrahim = reserve(store, Party("Ibrahim", 7), eight))
  val second = Thread(() => toCastellanos = reserve(store, Party("Castellanos", 8), eight))
  first.start()
  Thread.sleep(100)
  second.start()
  first.join()
  second.join()

  println(reply("Ibrahim", toIbrahim))
  println(reply("Castellanos", toCastellanos))

The two vars start as None and need their type written out, because None alone doesn’t tell the compiler what they’ll hold later. Run lastTable and it prints the same four lines as in chapter 4, with the double booking still there.

Restart the REPL, repeat the imports and try bestTable on tonight’s bookings:

scala> val eight = LocalTime.of(20, 0)
val eight: LocalTime = 20:00

scala> bestTable(Party("Silva", 2), eight, tonight)
val res0: Option[Table] = Some(Table(number = 1, seats = 2))

scala> bestTable(Party("Achterberg", 5), eight, tonight)
val res1: Option[Table] = None

scala> bestTable(Party("Zielinski", 9), eight, tonight)
val res2: Option[Table] = None

The Silvas get table 1 and the other two parties get None. For the party of five, the tables that would seat them are taken at eight. For the party of nine, no table in the restaurant is big enough at any time. The staff would say different things to those two callers, and None can’t tell them which is which. Use an Option when nobody will ask why the value is missing. Chapter 9 covers the case where the caller needs to know.

5.5. Nulls from Java

Scala has null because it runs on the JVM and calls Java. Scala’s own library almost never returns null, but Java methods do. This session doesn’t use the project, so start the REPL with plain scala repl:

scala> val port = System.getenv("FULLY_BOOKED_PORT")
val port: String = null

scala> Option(port)
val res0: Option[String] = None

scala> Option("8080")
val res1: Option[String] = Some("8080")

scala> Some(port)
val res2: Some[String] = Some(null)

scala> sys.env.get("FULLY_BOOKED_PORT")
val res3: Option[String] = None

scala> sys.env.get("FULLY_BOOKED_PORT").flatMap(_.toIntOption).getOrElse(8080)
val res4: Int = 8080

Wrapping a value in Option(...) turns null into None. Some(...) doesn’t check, so Some(port) came out as Some(null). If a Java method’s documentation says it can return null, put the call in Option(...). System.getenv is one of those methods, returning null for a variable that isn’t set. Left unwrapped, port would throw a NullPointerException at a call such as port.length.

The environment is also available as a Scala Map called sys.env, where get gives an Option directly. The last line reads a setting, parses it and falls back to 8080, whether the variable is missing or holds something that isn’t a number.

The compiler can check for nulls too, if you give it the option -Yexplicit-nulls. null then stops being a valid String in Scala code. A nullable value has to be declared as String | Null. Types written with that |, which chapter 2 read as “either”, are called union types. The check doesn’t cover what a Java method returns, so those values still need wrapping in Option(...). The option is off by default and the documentation lists it as experimental.

Java methods that return Optional need one import:

scala> import scala.jdk.OptionConverters.*

scala> val firstBig = java.util.List.of(4, 2, 8).stream().filter(_ > 5).findFirst()
val firstBig: java.util.Optional[Int] = Optional[8]

scala> firstBig.toScala
val res0: Option[Int] = Some(8)

scala> Option("8080").toJava
val res1: java.util.Optional[String] = Optional[8080]

The import brings in toScala, which turns an Optional into an Option, and toJava, which goes the other way. Convert where the Java value arrives and use Option everywhere after that.

5.6. Exercises

Solutions are in the book’s downloadable code, under ch05/exercises.

  1. Write largestParty(bookings: List[Booking]): Option[Booking], the biggest party among the bookings that hold a table. Try it on tonight and on List().

  2. The staff type a table number. Write tableNumbered(input: String): Option[Table], which gives None both for text that isn’t a number and for a number that isn’t one of the six tables.

  3. Write nextArrival(now: LocalTime, bookings: List[Booking]): Option[Booking], the next party due at or after now among the bookings that hold a table.

  4. Callers don’t always get the capitals right. Write a version of findBooking that tries an exact match first and then one that ignores case. orElse takes a second Option to use when the first is None, and Java’s equalsIgnoreCase works on a Scala String.

  5. get on a java.util.HashMap returns null for a missing key. Given a java.util.Map[String, String] of allergy notes keyed by party name, write allergy(name: String, notes: java.util.Map[String, String]): Option[String].

  6. Predict each result, then try it in the REPL: Some(4).map(_ * 2), Some(4).filter(_ > 5), Some(Some(4)).flatten and List(Some(1), None, Some(3)).flatten. flatten removes one layer of nesting, which is the second half of what flatMap does.

6. Testing with MUnit

So far we’ve checked Fully Booked by running it and reading what it printed. Reading output won’t tell us when a change breaks something that used to work. There’s also the flaw in freeAt that we’ve known about since chapter 4. We’ll catch it with a test and then fix it.

Scala’s standard library has no test framework, but the scala command can run several. I’ve chosen MUnit, which is small and will look familiar if you’ve used JUnit. ScalaTest is older and you’re likely to meet it in existing code, so there’s a section on it before the exercises.

6.1. A first test

MUnit is the first library we’ve needed that comes with neither Scala nor the JDK. Add a line to project.scala:

//> using test.dep org.scalameta::munit:1.3.6

If you’ve used Maven you’ll recognise the three parts as a group, an artifact and a version. A Scala library is published separately for each major version of Scala, with that version in the artifact’s name. The double colon asks for the build that matches our version of Scala, munit_3. A Java library is written with single colons throughout. Other people’s build files sometimes have a second double colon, before the version. That one only matters when Scala is being compiled for a platform other than the JVM. Writing test.dep in place of plain dep makes the library available to test code only.

Now create party.test.scala:

package fullybooked

class PartyTests extends munit.FunSuite:
  test("a party of four stays ninety minutes"):
    assertEquals(Party("Abara", 4).stayMinutes, 90)

  test("a party of five stays two hours"):
    assertEquals(Party("Okonkwo", 5).stayMinutes, 120)

  test("deposits start at six people"):
    assertEquals(Party("Okonkwo", 5).deposit, BigDecimal(0))
    assertEquals(Party("Haddad", 6).deposit, BigDecimal(60))

The .test.scala ending marks the file as test code. The scala command compiles such files separately. They can use everything in the project, but scala run ignores them and the main code can’t refer to them.

A suite of tests is a class that extends munit.FunSuite. Each call to test adds one test to it, with a name and a body. The body is the indented lines after the colon. Older code, and most of MUnit’s own documentation, puts it in braces, as in test("...") { ... }. Chapter 7 explains how a method can take a block of code as an argument.

The first argument of assertEquals is the value the code produced and the second is the value you expected. JUnit has them the other way round. If a failure message ever seems to be backwards, check the order.

The third test compares a deposit worked out as BigDecimal("10.00") * 6 with BigDecimal(60). The two are equal because Scala’s BigDecimal compares by value, where Java’s equals would also compare the scale and call 60.00 and 60 different.

Run the tests with scala test .:

$ scala test .
Test run fullybooked.PartyTests started
fullybooked.PartyTests: finished 0.01s
Test run fullybooked.PartyTests finished: 0 failed, 0 ignored, 3 total 0.01s

MUnit doesn’t say much when everything passes. The times will be different on your machine and on every run. In this chapter I’ve replaced them with round figures.

On Java 25 you’ll also get four lines of WARNING before any of that, about a method in sun.misc.Unsafe. MUnit 1.3.6 was compiled with Scala 3.3, and code from that compiler calls methods of an internal JDK class that Java now warns about and intends to remove. Other libraries compiled with Scala 3.3 or earlier print the same warning. The tests still run properly, so we’ll switch the warning off. Add a fourth line to project.scala, which passes an option to the JVM that runs the tests:

//> using test.javaOpt --sun-misc-unsafe-memory-access=allow

6.2. A test that fails

The flaw in freeAt is that it only asks whether a table is free at one moment. At eight o’clock it offers table 3, although the Moreaus have that table from quarter past. bestTable is built on freeAt and has the same problem. Before fixing it, we’ll write a test that shows it. Create availability.test.scala:

package fullybooked

import java.time.LocalTime

class AvailabilityTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)
  val moreau = Booking(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15), Status.Confirmed)

  test("the smallest table that fits is offered"):
    assertEquals(bestTable(Party("Silva", 2), eight, List(moreau)), Some(Table(1, 2)))

  test("a table that is booked during the stay is not offered"):
    assertEquals(bestTable(Party("Silva", 4), eight, List(moreau)), Some(Table(4, 4)))

The suite makes its own booking and leaves tonight alone. I keep test data in the test file. A test that depends on tonight.scala breaks when somebody edits that list for an unrelated reason, and anyone reading the test has to open a second file to see why it expects what it does.

Both tests ask for a table at eight with only the Moreau booking in the list. A party of two should get table 1. For a party of four, tables 3 and 4 are the smallest that fit, but the Moreaus will want table 3 fifteen minutes later, so the right answer is table 4. --test-only runs a single suite:

$ scala test . --test-only fullybooked.AvailabilityTests
Test run fullybooked.AvailabilityTests started
==> X fullybooked.AvailabilityTests.a table that is booked during the stay is not offered 0.01s munit.ComparisonFailException: .../fully-booked/availability.test.scala:13
12:  test("a table that is booked during the stay is not offered"):
13:    assertEquals(bestTable(Party("Silva", 4), eight, List(moreau)), Some(Table(4, 4)))
values are not the same
=> Obtained
Some(
  value = Table(
    number = 3,
    seats = 4
  )
)
=> Diff (- expected, + obtained)
   value = Table(
-    number = 4,
+    number = 3,
     seats = 4
    at munit.FunSuite.assertEquals(FunSuite.scala:12)
    at fullybooked.AvailabilityTests.$init$$$anonfun$2(availability.test.scala:13)
fullybooked.AvailabilityTests: finished 0.01s
Test run fullybooked.AvailabilityTests finished: 1 failed, 0 ignored, 2 total 0.01s

The second test got table 3 where there should have been table 4. MUnit shows that in the diff, where - marks what the test expected and + what it got. Above the diff are the lines of the test and the value the code produced. The line starting ==> X names the failed test and says where it is. I’ve shortened the path of the file, which MUnit prints in full. At the end the scala command prints Error: munit tests failed. and finishes with a non-zero exit status, which is what a build server checks.

The fix is to compare the party’s whole stay with each booking’s. Two stays overlap when each starts before the other ends. If you did exercise 7 in chapter 4 you’ve written freeFor already, and only need to move it into availability.scala. Otherwise add it there, above bestTable:

def freeFor(party: Party, time: LocalTime, bookings: List[Booking]): List[Table] =
  val end = time.plusMinutes(party.stayMinutes)
  val taken = bookings
    .filter(holdsTable)
    .filter(booking => time.isBefore(booking.leaveBy) && booking.time.isBefore(end))
    .map(_.table)
    .toSet
  Table.all.filterNot(taken.contains).filter(_.seats >= party.size)

Then replace bestTable with a version that uses it:

def bestTable(party: Party, time: LocalTime, bookings: List[Booking]): Option[Table] =
  freeFor(party, time, bookings).minByOption(_.seats)

Keep freeAt, because lastTable still prints with it. Run the tests again and both pass.

Add three more tests to the end of AvailabilityTests:

  test("a table is free again at the minute its party leaves"):
    val nakamura = Booking(Party("Nakamura", 2), Table(1, 2), LocalTime.of(18, 30), Status.Seated)
    assertEquals(nakamura.leaveBy, eight)
    assert(freeFor(Party("Silva", 2), eight, List(nakamura)).contains(Table(1, 2)))

  test("a cancelled booking does not hold its table"):
    val cancelled = moreau.copy(status = Status.Cancelled)
    assertEquals(bestTable(Party("Silva", 4), eight, List(cancelled)), Some(Table(3, 4)))

  test("nobody gets a table when the party is too big"):
    assertEquals(bestTable(Party("Zielinski", 9), eight, List()), None)

The first of them opens with an assertEquals that checks the Nakamuras really are due to leave at eight. If the length of a stay ever changes, that line fails first and says why. The test’s second assertion is a plain assert, which takes any Boolean. Use assertEquals when you’re comparing two values, because a failed assert can only report that something was false.

To see every test listed, pass -v to MUnit. Anything after -- goes to the test framework and not to the scala command:

$ scala test . --test-only fullybooked.AvailabilityTests -- -v
Test run fullybooked.AvailabilityTests started
fullybooked.AvailabilityTests:
fullybooked.AvailabilityTests.the smallest table that fits is offered started
  + the smallest table that fits is offered 0.01s
fullybooked.AvailabilityTests.a table that is booked during the stay is not offered started
  + a table that is booked during the stay is not offered 0.01s
fullybooked.AvailabilityTests.a table is free again at the minute its party leaves started
  + a table is free again at the minute its party leaves 0.01s
fullybooked.AvailabilityTests.a cancelled booking does not hold its table started
  + a cancelled booking does not hold its table 0.01s
fullybooked.AvailabilityTests.nobody gets a table when the party is too big started
  + nobody gets a table when the party is too big 0.01s
fullybooked.AvailabilityTests: finished 0.01s
Test run fullybooked.AvailabilityTests finished: 0 failed, 0 ignored, 5 total 0.01s

6.3. A new store for each test

The store is the one part of Fully Booked that changes as the program runs. That makes it awkward to test, because a booking added by one test mustn’t still be there when the next one starts. MUnit can make a new value for each test with what it calls a fixture. Create store.test.scala:

package fullybooked

import java.time.LocalTime

class StoreTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)

  val emptyStore = FunFixture[BookingStore](
    setup = _ => InMemoryBookingStore(),
    teardown = _ => ()
  )

  emptyStore.test("a new store has no bookings"): store =>
    assertEquals(store.all, List())

  emptyStore.test("at finds the bookings for a time"): store =>
    store.add(Booking(Party("Lindqvist", 7), Table(6, 8), eight))
    store.add(Booking(Party("Fitzgerald", 2), Table(2, 2), LocalTime.of(21, 15)))
    assertEquals(store.at(eight).map(_.party.name), List("Lindqvist"))

  emptyStore.test("reserve saves a confirmed booking"): store =>
    val made = reserve(store, Party("Silva", 2), eight)
    assertEquals(made.map(_.table), Some(Table(1, 2)))
    assertEquals(store.all.map(_.status), List(Status.Confirmed))

A FunFixture is built from two lambdas, and the type in square brackets is the type of value it provides. Before each test MUnit runs setup to make that value, and afterwards it runs teardown to clear up. It hands setup some details of the test and hands teardown the value that setup made. We need neither here, which is why each lambda takes its argument as _. A test declared with emptyStore.test gets the new store as a parameter: its body starts store => after the colon and carries on beneath.

Our teardown does nothing, a sign that we could have managed without a fixture: val store = InMemoryBookingStore() as the first line of each test would have done. I use a fixture when there’s something to clear up afterwards, such as a temporary file or a database connection. If you’re used to @BeforeEach in JUnit, MUnit also has beforeEach and afterEach methods you can override.

Run all the tests. Your suites may come out in a different order from mine:

$ scala test .
Test run fullybooked.PartyTests started
fullybooked.PartyTests: finished 0.01s
Test run fullybooked.PartyTests finished: 0 failed, 0 ignored, 3 total 0.01s
Test run fullybooked.StoreTests started
fullybooked.StoreTests: finished 0.3s
Test run fullybooked.StoreTests finished: 0 failed, 0 ignored, 3 total 0.3s
Test run fullybooked.AvailabilityTests started
fullybooked.AvailabilityTests: finished 0.01s
Test run fullybooked.AvailabilityTests finished: 0 failed, 0 ignored, 5 total 0.01s

The StoreTests suite took three tenths of a second, nearly all of it the Thread.sleep(300) inside reserve. One sleeping test is tolerable, but a few hundred would make a suite that takes minutes, and a suite that slow tends not to get run before every commit. Chapter 21 shows how to test code that waits without the test taking that long.

6.4. Running some of the tests

The --test-only option takes a pattern as well as a full name, so --test-only 'fullybooked.Avail*' works. To choose tests by their names, give MUnit a pattern after the --:

$ scala test . --test-only fullybooked.PartyTests -- '*deposit*' -v
Test run fullybooked.PartyTests started
fullybooked.PartyTests:
fullybooked.PartyTests.deposits start at six people started
  + deposits start at six people 0.01s
fullybooked.PartyTests: finished 0.01s
Test run fullybooked.PartyTests finished: 0 failed, 0 ignored, 1 total 0.01s

While you’re working on one test you can also mark it in the code. A test declared as test("...".only) is the only one in its suite that runs, and .ignore in the same place skips a test. Take either out again before you commit. Chapter 11 explains how a String comes to have methods called only and ignore.

With --watch on the end, scala test . re-runs the tests every time you save. Metals and IntelliJ both show a run button beside each test.

6.5. What’s worth testing

Some tests you might write in Java aren’t needed here. No method in Fully Booked accepts or returns null. A match on Status that forgets a case is reported by the compiler, and so is an Option[Table] passed where a Table is wanted. That last check holds inside a test as well: assertEquals(bestTable(...), Table(1, 2)) doesn’t compile, because MUnit refuses to compare an Option[Table] with a Table. There’s also no point testing equals, toString or copy on a case class, which the compiler generates.

Nothing in the types says that a party of four stays ninety minutes and a party of five stays two hours, or that a table is free again at the minute its party leaves. I’d write tests for those first: every rule with a number in it, tried on both sides of the number.

A test for the last-table race would have to start two threads and hope they collide, so it would only catch the bug on some runs. I’ve left it until chapter 21, where the race is run against a real database.

6.6. ScalaTest

MUnit dates from 2020 and ScalaTest from Scala’s early years. You might also come across specs2, uTest, Weaver and ZIO Test, or plain JUnit. The two tests of Party below are written for ScalaTest. You don’t need to type them in. They’re in the book’s downloadable code under ch06/scalatest, in a directory of their own with org.scalatest::scalatest:3.2.20 as the test dependency:

package fullybooked

import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers

class PartySpec extends AnyFlatSpec, Matchers:
  "A party of four" should "stay ninety minutes" in {
    Party("Abara", 4).stayMinutes shouldBe 90
  }

  "A party of six" should "pay a deposit of ten pounds a head" in {
    Party("Haddad", 6).deposit shouldBe BigDecimal(60)
  }

The style here, AnyFlatSpec, is one of several in ScalaTest. It has you write each test as a sentence, in which should, in and shouldBe are ordinary methods called without dots or parentheses. shouldBe comes from Matchers, along with a long list of assertions that read like it. Another style, AnyFunSuite, looks much like MUnit. Two ScalaTest codebases written in different styles can look quite unlike each other, so when you join one, follow the style that’s already there.

These tests run with the same command as MUnit’s, scala test ..

6.7. Exercises

Solutions are in the book’s downloadable code, under ch06/exercises.

  1. Write three tests for findBooking: a name that’s in the list, a name that isn’t, and a name with the wrong capitals.

  2. Test leaveBy for a party of four and a party of five, both arriving at seven.

  3. intercept[IllegalArgumentException]: followed by an indented block passes when the block throws that exception and fails when it doesn’t. Use it to test that Status.valueOf("Gone") is rejected.

  4. Test label for a party of six, once with a confirmed booking and once with a requested one.

  5. The Patels, two people on table 1, ask to swap with the Abaras, four people on table 4. Write a test that says what swapTables should return, watch it fail, then fix swapTables. In a for laid out over several lines, an if can go on a line of its own after the <- clauses.

  6. Mark one test in AvailabilityTests with .only and run the suite with -v. Then change it to .ignore. What does MUnit print each time?

Part II: Getting the compiler to do the work

7. Functions as values

The restaurant’s owner wants deposits to depend on the night. Fridays and Saturdays should be £15 a head for any party of four or more, and on New Year’s Eve everybody pays £25 a head. At the moment there’s one rule, written into a method on Party with its numbers inside it.

We could add a method for each kind of night and an if to choose between them. I’d make the rule a value instead: a function that we can store in a val, pass to a method and build out of other rules.

7.1. A rule as a value

Create deposits.scala:

package fullybooked

type DepositRule = Party => BigDecimal

def perHead(amount: BigDecimal, from: Int): DepositRule =
  party => if party.size >= from then amount * party.size else BigDecimal(0)

val weekday: DepositRule = perHead(BigDecimal(10), from = 6)
val weekend: DepositRule = perHead(BigDecimal(15), from = 4)
val newYearsEve: DepositRule = perHead(BigDecimal(25), from = 1)

The type line gives the name DepositRule to Party => BigDecimal, the type of any function that takes a Party and returns a BigDecimal. The compiler has been inferring types like that for most of the lambdas we’ve written. It treats the name and the longer spelling as the same type, so the name only saves us from repeating it.

The lambda that perHead returns can still read amount and from long after perHead itself has returned. A function that holds on to values from the place it was written is a closure. So weekday, weekend and newYearsEve are three different functions, each with its own amount and threshold inside it. To use one, write the party in parentheses after it, as in weekday(party).

In Java the type would be Function<Party, BigDecimal> and the call rule.apply(party). Underneath, Scala does much the same: weekday is an object, and weekday(party) runs its apply method.

Delete the deposit method from Party in model.scala, now that the rule lives in deposits.scala. The test called “deposits start at six people” in party.test.scala no longer compiles, so delete that as well. Its replacement goes in a new file, deposits.test.scala:

package fullybooked

import java.time.LocalTime

class DepositTests extends munit.FunSuite:
  val five = Party("Okonkwo", 5)
  val six = Party("Haddad", 6)

  test("weekday deposits start at six people"):
    assertEquals(weekday(five), BigDecimal(0))
    assertEquals(weekday(six), BigDecimal(60))

  test("weekend deposits are higher and start at four"):
    assertEquals(weekend(Party("Moreau", 3)), BigDecimal(0))
    assertEquals(weekend(five), BigDecimal(75))

Run scala test . and all four suites should pass.

7.2. Handing a rule to a method

To add up tonight’s deposits we need to know which rule is in force, so the method that does it takes the rule as a parameter. Add this to the end of deposits.scala:

def depositsDue(bookings: List[Booking], rule: DepositRule): BigDecimal =
  bookings.filter(holdsTable).map(booking => rule(booking.party)).sum

A caller can pass any function from Party to BigDecimal as rule. In Java’s strategy pattern the caller would pass an object of a class that implements an interface. A method like depositsDue that takes a function is called a higher-order function, and so is one that returns a function, as perHead does. Add a test for depositsDue to the end of DepositTests:

  test("deposits are only due from bookings that hold a table"):
    val eight = LocalTime.of(20, 0)
    val bookings = List(
      Booking(six, Table(5, 6), eight, Status.Confirmed),
      Booking(Party("Lindqvist", 7), Table(6, 8), eight, Status.Cancelled)
    )
    assertEquals(depositsDue(bookings, weekday), BigDecimal(60))

To see what each rule charges, add another @main to the end of main.scala:

@main def deposits(): Unit =
  val rules = List("Weekday" -> weekday, "Weekend" -> weekend, "New Year's Eve" -> newYearsEve)
  for (name, rule) <- rules do
    val amounts = List(2, 4, 6, 8).map(size => rule(Party("Example", size)))
    println(s"$name: $amounts, due tonight ${depositsDue(tonight, rule)}")
$ scala run . --main-class fullybooked.deposits
Weekday: List(0, 0, 60, 80), due tonight 70
Weekend: List(0, 60, 90, 120), due tonight 240
New Year's Eve: List(50, 100, 150, 200), due tonight 575

The for takes each pair in the list apart into name and rule.

We’ve been passing methods like holdsTable to filter since chapter 4. Strictly, a method isn’t a value: it belongs to the class, object or file it’s declared in. Those calls work because the compiler turns a method into a function wherever a function is expected, so a rule written as def flatTen(party: Party): BigDecimal could be passed to depositsDue too. I write def by default and use a function value when other code has to build it, as perHead builds these rules.

7.3. Rules made from rules

Bank holidays get the weekend rule or £20 a head for parties of eight and over, whichever is more, up to a limit of £150. We can build that out of the rules we already have. Add these to the end of deposits.scala:

def cappedAt(limit: BigDecimal): BigDecimal => BigDecimal =
  amount => amount.min(limit)

def higherOf(first: DepositRule, second: DepositRule): DepositRule =
  party => first(party).max(second(party))

val bankHoliday: DepositRule =
  higherOf(weekend, perHead(BigDecimal(20), from = 8)).andThen(cappedAt(BigDecimal(150)))

In bankHoliday, higherOf makes one rule from two. andThen hands that rule’s answer to the function cappedAt returned, which lowers the amount to the limit if it’s over. Every one-argument function has an andThen method. The call only compiles if the first function returns what the second takes. There’s a compose as well, which joins two functions in the opposite order, but I stick to andThen because it reads in the order things happen.

Take a party of eight. The weekend rule says £120, the other says £160, the higher is £160 and the cap brings it down to £150. Put that in a test at the end of DepositTests:

  test("a bank holiday takes the higher rule and stops at 150"):
    assertEquals(bankHoliday(five), BigDecimal(75))
    assertEquals(bankHoliday(Party("Castellanos", 8)), BigDecimal(150))

7.4. Longer lambdas

Cancellations have rules as well: someone who cancels on the day gets the whole deposit back with six hours’ notice, half with two, and otherwise nothing. This rule needs two arguments, the deposit and the hours of notice, and a body of several lines. Put import java.time.{Duration, LocalTime} under the package line of deposits.scala, then add this to the end of the file:

type RefundRule = (BigDecimal, Long) => BigDecimal

val sameDay: RefundRule = (deposit, hoursNotice) =>
  if hoursNotice >= 6 then deposit
  else if hoursNotice >= 2 then deposit / 2
  else BigDecimal(0)

def refund(booking: Booking, now: LocalTime, deposit: DepositRule, rule: RefundRule): BigDecimal =
  val notice = Duration.between(now, booking.time).toHours
  rule(deposit(booking.party), notice)

The two arguments of a RefundRule go in parentheses in its type. The hours are a Long because toHours, a method on Java’s Duration, returns one. In sameDay the body starts on the line after the => and is indented, the way a method’s body follows its =. refund then needs both kinds of rule: deposit to work out what was paid and rule to decide how much of it comes back.

The test for refund goes at the end of DepositTests:

  test("a same-day cancellation gets half back with two hours' notice"):
    val booking = Booking(six, Table(5, 6), LocalTime.of(20, 0), Status.Confirmed)
    assertEquals(refund(booking, LocalTime.of(13, 0), weekday, sameDay), BigDecimal(60))
    assertEquals(refund(booking, LocalTime.of(17, 30), weekday, sameDay), BigDecimal(30))
    assertEquals(refund(booking, LocalTime.of(19, 0), weekday, sameDay), BigDecimal(0))

In chapter 5 reserve stayed a match because its second case was too long for a one-line lambda. A lambda with an indented body, like the one in sameDay, is long enough, so the match can become a map. Replace reserve in availability.scala:

def reserve(store: BookingStore, party: Party, time: LocalTime): Option[Booking] =
  bestTable(party, time, store.all).map: table =>
    Thread.sleep(300)
    val booking = Booking(party, table, time, Status.Confirmed)
    store.add(booking)
    booking

The colon after map says that its argument comes next: the lambda’s parameter and => on the same line, then the body on the indented lines beneath. We’ve been writing that colon after test(...) since chapter 6. Code written with braces has .map { table => ... } here, with the closing brace after the last line.

Run the tests: the one in StoreTests that calls reserve should still pass.

7.5. Several parameter lists

A method can declare more than one parameter list. In chapter 4, foldLeft took its arguments in two pairs of parentheses, foldLeft(0)((total, booking) => ...). Its reason goes back to Scala 2, which worked out types one list at a time. A 0 in a list of its own told the compiler that the running total was an Int before it reached the lambda. Scala 3 can usually work that out within a single list, but foldLeft still has two.

The kitchen takes bookings from noon until half past nine, and any code that makes a booking should check that first. A method with two parameter lists can do the check and take the booking code as its second list, written after a colon. Add the hours and the method to the end of availability.scala:

val opens = LocalTime.of(12, 0)
val lastOrders = LocalTime.of(21, 30)

def whenOpen(time: LocalTime)(action: => String): String =
  if time.isBefore(opens) || time.isAfter(lastOrders) then s"The kitchen is closed at $time"
  else action

The type of action is => String, with nothing to the left of the arrow, which makes it a by-name parameter. Scala works out an ordinary argument before it calls the method, but it hands a by-name argument over as code. A Java method would declare a Supplier<String> for this, and its caller would write () -> in front of the code. In Scala the caller writes only the code, and it runs each time the method’s body uses action, or not at all if the body never does.

Add a @main to the end of main.scala to try it:

@main def openingHours(): Unit =
  val store = InMemoryBookingStore()
  val party = Party("Silva", 2)
  for time <- List(LocalTime.of(11, 0), LocalTime.of(20, 0)) do
    val answer = whenOpen(time):
      println(s"Looking for a table at $time")
      reply(party.name, reserve(store, party, time))
    println(answer)
$ scala run . --main-class fullybooked.openingHours
The kitchen is closed at 11:00
Looking for a table at 20:00
Silva: table 1 at 20:00

At eleven the block after whenOpen(time): didn’t run: nothing was printed from inside it and no table was reserved. At eight it ran, with its last line as the value of action. MUnit’s test takes the name in one list and a by-name body in the next, which lets it collect every test in a suite first and run the bodies later.

A method with two lists can be called with only the first. If perHead were def perHead(amount: BigDecimal, from: Int)(party: Party): BigDecimal, then perHead(BigDecimal(15), 4) would be a function that still needs a party, in other words a DepositRule. A method written that way is said to be curried.

7.6. Lambdas made of cases

Each afternoon someone rings round the confirmed bookings to check that the guests are still coming. The list to ring from needs the time, name and number of every confirmed booking that has a phone number. We could build it with a filter and then a map, but collect does both at once. Add this to the end of desk.scala:

def toReconfirm(bookings: List[Booking]): List[String] =
  bookings.collect:
    case Booking(Party(name, _, Some(phone)), _, time, Status.Confirmed) =>
      s"$time $name on $phone"

Add one more @main to the end of main.scala:

@main def reconfirm(): Unit =
  for line <- toReconfirm(tonight) do println(line)
$ scala run . --main-class fullybooked.reconfirm
19:30 Okonkwo on 07700 900141
20:00 Lindqvist on 07700 900152
20:15 Moreau on 07700 900164

The lambda we gave collect is a single case with a pattern like those in a match, though it could have had several. The pattern asks for a confirmed booking with Some(phone) inside its Party. collect skips every booking that doesn’t fit and applies the case to the rest.

A lambda that has an answer for some inputs and not for others is a partial function, and collect is built to take one. Give map the same case and the compiler warns that the match may not be exhaustive, then the program throws a MatchError on the first booking in the list, the Haddads’ cancelled one. In library signatures, including the one for collect, the type is written PartialFunction.

You may also meet case lambdas in older code that takes tuples apart. For chapter 4’s (hour, bookings) => ..., Scala 2 needed { case (hour, bookings) => ... }.

7.7. Exercises

Solutions are in the book’s downloadable code, under ch07/exercises.

  1. Write flat(amount: BigDecimal): DepositRule, a rule that charges every party the same deposit whatever its size, and test it.

  2. Write discounted(rule: DepositRule, percent: Int): DepositRule. A weekend party of eight with 20% off should pay £96.

  3. Write strictest(rules: List[DepositRule]): DepositRule, which charges the highest amount that any rule in the list asks for.

  4. Write depositsDueByFold, a version of depositsDue that uses foldLeft with its lambda after a colon. A lambda with two parameters starts (total, booking) =>.

  5. Write twice(action: => Unit): Unit, which uses action two times. Test it with a block that adds one to a var. Then change the parameter’s type to plain Unit and work out why the test fails.

  6. Write bigAndConfirmed(bookings: List[Booking]): List[String], the names of confirmed parties of four or more, using one collect. A case in a lambda can have an if guard.

  7. label still prints “deposit taken” for any confirmed party of six or more, whatever the night. Give it a DepositRule parameter, so that it says so only when the rule charges that party something, and fix the code that calls it.

8. Modelling with types

What time did the Nakamuras sit down? Why did the Haddads cancel? Our bookings can’t say, because a Status is one of six names with nowhere to keep a time or a reason.

A Java developer might add two fields to Booking, seatedAt and cancellationReason, both null most of the time, with a comment saying which statuses fill them in. In Scala each status can carry its own data. The compiler then checks that every seated booking has a time and every cancelled one a reason.

8.1. Cases that carry data

Replace Status in model.scala:

enum Status:
  case Requested, Confirmed, Completed, NoShow
  case Seated(at: LocalTime)
  case Cancelled(reason: String)

The Seated case now has the time the party sat down and Cancelled has a reason, while the other four statuses are still plain names. A case with fields behaves like a small case class: you build one with Status.Cancelled("flu"), two of them with the same reason are equal, and a pattern can take one apart.

Compiling the project now gives errors in five places, though the compiler may not report all five at once. Each is a line that uses Status.Seated or Status.Cancelled with no argument, where it now needs a time or a reason. Three of them are the first three bookings in tonight.scala. Replace those with:

  Booking(Party("Haddad", 6), Table(5, 6), LocalTime.of(18, 0),
    Status.Cancelled("rang to cancel")),
  Booking(Party("Nakamura", 2), Table(2, 2), LocalTime.of(18, 30),
    Status.Seated(LocalTime.of(18, 25))),
  Booking(Party("Abara", 4), Table(4, 4), LocalTime.of(18, 45),
    Status.Seated(LocalTime.of(18, 50))),

In walkIn, in the Booking companion, Status.Seated becomes Status.Seated(now). The fifth is the line inside the for in lastTable. Change it to seat the regulars with walkIn: store.add(Booking.walkIn(Party("Regular", table.seats), table, eight)).

Compile again:

$ scala compile .
[warn] ./desk.scala:3:40
[warn] match may not be exhaustive.
[warn]
[warn] It would fail on pattern case: fullybooked.Status.Seated(_), fullybooked.Status.Cancelled(_)
[warn] def describe(status: Status): String = status match
[warn]                                        ^^^^^^
[warn] ./desk.scala:6:8
[warn] Unreachable case
[warn]   case Status.Seated    => "at the table"
[warn]        ^^^^^^^^^^^^^
[warn] ./desk.scala:8:8
[warn] Unreachable case
[warn]   case Status.Cancelled => "cancelled"
[warn]        ^^^^^^^^^^^^^^^^

The first warning lists the two cases that describe no longer covers. The other two call its Status.Seated and Status.Cancelled lines unreachable. Written with no arguments, Status.Seated is the companion object that builds Seated values. No status is ever equal to that, so neither that line nor the Status.Cancelled one can ever match. Replace describe in desk.scala:

def describe(status: Status): String = status match
  case Status.Requested         => "waiting to be confirmed"
  case Status.Confirmed         => "confirmed"
  case Status.Seated(at)        => s"at the table since $at"
  case Status.Completed         => "been and gone"
  case Status.Cancelled(reason) => s"cancelled ($reason)"
  case Status.NoShow            => "never turned up"

The pattern Status.Seated(at) matches any seated status and gives its time the name at.

holdsTable got no warning at all, yet it’s the one with the bug. Its pattern Status.Confirmed | Status.Seated stopped matching seated bookings, because that Status.Seated is the companion too. Its second case is case _ => false, so every party that’s sitting at a table is now reported as not holding it. freeFor would offer their table to someone else. The compiler said nothing: a catch-all silences the missing-case warning, and an alternative inside a | that can never match isn’t reported as unreachable. Replace holdsTable with a version that names every status:

def holdsTable(booking: Booking): Boolean = booking.status match
  case Status.Confirmed | Status.Seated(_) => true
  case Status.Requested | Status.Completed | Status.NoShow | Status.Cancelled(_) => false

When a seventh status is added, the compiler will warn about this version until someone decides which side the new status belongs on. I don’t use case _ in a match over an enum of my own unless the enum is very large.

Three lines in the tests need a time or a reason as well before they’ll compile. In availability.test.scala, build the Nakamura booking with Booking.walkIn(Party("Nakamura", 2), Table(1, 2), LocalTime.of(18, 30)) and change the cancelled status to Status.Cancelled("flu"). Make the same change to the cancelled booking in deposits.test.scala. If you kept a test of Status.valueOf from chapter 6’s exercises, delete it, because Status no longer has that method. Then run the tests, and frontDesk:

$ scala run . --main-class fullybooked.frontDesk
18:00 table 5: Haddad, party of 6, cancelled (rang to cancel)
18:30 table 2: Nakamura, party of 2, at the table since 18:25
18:45 table 4: Abara, party of 4, at the table since 18:50
19:30 table 5: Okonkwo, party of 5, confirmed
19:30 table 1: Patel, party of 2, waiting to be confirmed
20:00 table 6: Lindqvist, party of 7 (deposit taken), confirmed
20:15 table 3: Moreau, party of 3, confirmed
21:15 table 2: Fitzgerald, party of 2, confirmed

Like a Java enum, a Scala enum normally has values and valueOf. Status lost both when two of its cases gained fields, because there’s no longer a fixed list of values to return.

8.2. What can happen to a booking

A status is where a booking has got to. It gets there through events: the restaurant confirms the booking, and the party arrives, leaves, cancels or never shows up. Create events.scala:

package fullybooked

import java.time.LocalTime

sealed trait Event
case object Confirm extends Event
case class Arrive(at: LocalTime) extends Event
case object Leave extends Event
case class Cancel(reason: String) extends Event
case object FailToArrive extends Event

An enum is shorthand for this: a trait and an implementation for each case. The two events with fields are case classes. The three without are case objects, each of them a single value. sealed means that anything extending Event directly has to be in this file. Java has had sealed interfaces since Java 17, where a permits list can name implementations in other files. The compiler then knows the full list and can warn about a match on an Event that misses a case.

In new Scala 3 code I’d write Event as an enum, which is exercise 2. I’ve written it out in full because enum arrived with Scala 3, so Scala 2 code usually has a sealed trait where Scala 3 would have an enum. A sealed trait is still needed in Scala 3 when one of the cases wants methods of its own, which an enum case can’t have.

Which events are allowed depends on the status, so the method that applies the rules takes both. Add it to events.scala:

def next(status: Status, event: Event): Option[Status] = (status, event) match
  case (Status.Requested, Confirm)      => Some(Status.Confirmed)
  case (Status.Confirmed, Arrive(at))   => Some(Status.Seated(at))
  case (Status.Confirmed, FailToArrive) => Some(Status.NoShow)
  case (Status.Seated(_), Leave)        => Some(Status.Completed)
  case (Status.Requested | Status.Confirmed, Cancel(reason)) =>
    Some(Status.Cancelled(reason))
  case _ => None

The match is on a tuple of the status and the event, with a case for each permitted move. A requested booking can be confirmed, a confirmed one can arrive or fail to, and so on down. In the case for Cancel, the | from chapter 3 sits inside a tuple pattern. Every other combination falls to the final case and gets None. I said I avoid case _, but there are thirty combinations here and “everything else isn’t allowed” is exactly what the rule says.

Give Booking a method that calls next with its own status, inside the class after leaveBy:

  def handle(event: Event): Option[Booking] =
    next(status, event).map(newStatus => copy(status = newStatus))

Another @main at the end of main.scala puts one booking through an evening:

@main def lifecycle(): Unit =
  var booking = Booking(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15))
  val events = List(Confirm, Arrive(LocalTime.of(20, 10)), Cancel("too noisy"), Leave)
  for event <- events do
    booking.handle(event) match
      case Some(changed) =>
        booking = changed
        println(s"$event: ${describe(booking.status)}")
      case None =>
        println(s"$event: not allowed while ${describe(booking.status)}")
$ scala run . --main-class fullybooked.lifecycle
Confirm: confirmed
Arrive(20:10): at the table since 20:10
Cancel(too noisy): not allowed while at the table since 20:10
Leave: been and gone

The Moreaus are confirmed, arrive and then try to cancel, which a seated party can’t do. Their booking is left as it was and the next event carries on from there. The events print as Confirm and Arrive(20:10) because case objects and case classes come with a toString.

Chapter 1 said that a cancelled booking must not be seated. Put that in a test, along with some of the other rules. Create events.test.scala:

package fullybooked

import java.time.LocalTime

class EventTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)
  val requested = Booking(Party("Moreau", 3), Table(3, 4), eight)

  test("a booking goes from requested to completed"):
    val events = List(Confirm, Arrive(eight), Leave)
    val end = events.foldLeft(Option(requested)): (booking, event) =>
      booking.flatMap(_.handle(event))
    assertEquals(end.map(_.status), Some(Status.Completed))

  test("a cancelled booking can't be seated"):
    val cancelled = requested.handle(Cancel("flu"))
    assertEquals(cancelled.map(_.status), Some(Status.Cancelled("flu")))
    assertEquals(cancelled.flatMap(_.handle(Arrive(eight))), None)

  test("a party at its table can't cancel"):
    val seated = requested.copy(status = Status.Seated(eight))
    assertEquals(seated.handle(Cancel("changed our minds")), None)

  test("seated parties hold their table"):
    assert(holdsTable(requested.copy(status = Status.Seated(eight))))

The first test folds a list of events over a booking. The fold starts from Option(requested), which has the type Option[Booking]. Some(requested) would have the narrower type Some[Booking], and the compiler would then reject a lambda that can return None. The lambda after the colon takes the booking so far and the next event. Because the lambda uses flatMap, the result becomes None at the first event that isn’t allowed and stays None to the end. The last test would have caught the bug in holdsTable.

At the moment next answers None without saying why, so a member of staff who taps “seat” on a cancelled booking can’t be told what’s wrong. Chapter 9 changes next to return the reason.

8.3. How far to take it

The rules in next are checked when the program runs. The compiler could check them if each state had a type of its own. With a method seat that only accepted a ConfirmedBooking, an attempt to seat a cancelled booking wouldn’t compile, and that rule would need no test.

A list of tonight’s bookings would then hold a mixture of those types, so every method that reads the list would have to check which type each one is. The store, the availability code and most of the tests would all grow. I’d accept that for a small piece of code where a mistake would do real damage. For Fully Booked I’d rather have a status with data and one function that holds the rules.

8.4. A type for phone numbers

Party("07700 900141", 5, Some("Okonkwo")) compiles. The name and the number are both strings, so the compiler can’t tell that they’ve been swapped. Nothing stops “ask for Dev” being stored as a phone number either. Create phone.scala:

package fullybooked

opaque type Phone = String

object Phone:
  def parse(text: String): Option[Phone] =
    val digits = text.filterNot(_.isWhitespace)
    if digits.length >= 10 && digits.forall(_.isDigit) then Some(digits) else None

opaque type Phone = String says that a Phone is a String and that only the code in this file is allowed to know it. Everywhere else Phone is a type of its own. A String isn’t accepted where a Phone is wanted, and a Phone has none of the methods of String. Other code has to get its phone numbers from Phone.parse, which drops the spaces and insists on at least ten digits. parse can call filterNot and forall on a String because a String works as a collection of characters. isWhitespace and isDigit are tests on a single character. Because parse checks what it’s given and nothing else can make a Phone, it’s called a smart constructor.

Change the first line of Party to use Phone:

case class Party(name: String, size: Int, phone: Option[Phone] = None):

In tonight.scala, replace the four bookings from Okonkwo to Moreau:

  Booking(Party("Okonkwo", 5, Phone.parse("07700 900141")), Table(5, 6),
    LocalTime.of(19, 30), Status.Confirmed),
  Booking(Party("Patel", 2, Phone.parse("07700 900129")), Table(1, 2),
    LocalTime.of(19, 30)),
  Booking(Party("Lindqvist", 7, Phone.parse("07700 900152")), Table(6, 8),
    LocalTime.of(20, 0), Status.Confirmed),
  Booking(Party("Moreau", 3, Phone.parse("07700 900164")), Table(3, 4),
    LocalTime.of(20, 15), Status.Confirmed),

The Some has gone because Phone.parse already returns an Option[Phone], the type the field wants. Both lookUp and reconfirm now print the numbers without their spaces, because that’s how parse keeps them. lookUp still compiles, although its getOrElse("none given") gives a String as the default for an Option[Phone]. Chapter 10 explains why that’s allowed. Outside its own file a Phone has only the methods that every value has, such as toString. Chapter 11 adds some.

At run time a Phone is an ordinary String with no wrapper object around it. The other way of doing this is a case class with one field and a private constructor, case class Phone private (digits: String). That form makes an object for every number. Scala 2 has no opaque type, so code written for it wraps the value in a class as well. Such a class sometimes ends in extends AnyVal, which asks the compiler to leave the wrapper out where it can.

Add a test file, phone.test.scala:

package fullybooked

class PhoneTests extends munit.FunSuite:
  test("spaces are dropped from a number"):
    assertEquals(Phone.parse("07700 900141").map(_.toString), Some("07700900141"))

  test("letters are not a phone number"):
    assertEquals(Phone.parse("ask for Dev"), None)

  test("a number that is too short is rejected"):
    assertEquals(Phone.parse("0770 900"), None)

A badly typed number now becomes None, as though the party had given no number at all. That’s acceptable for the list in tonight.scala, which we typed ourselves. A customer who has just entered a number should be told it’s wrong, and in chapter 9 they are.

8.5. Exercises

Solutions are in the book’s downloadable code, under ch08/exercises.

  1. Add an event Reinstate, which returns a cancelled booking to Requested. Test that it works and that a booking in any other status refuses it.

  2. Rewrite Event as an enum. What has to change in next, in lifecycle and in the tests?

  3. Write an opaque type PartySize over Int, in a file of its own, with a smart constructor PartySize.from(size: Int): Option[PartySize] that accepts sizes from 1 to Booking.largestParty.

  4. Callers from abroad give numbers like +44 7700 900141. Change Phone.parse to accept a leading + and keep it. startsWith and drop work on a String.

  5. Delete Status.NoShow from the second case of holdsTable and compile. Read the warning, then put it back. What would have happened with case _ => false on that line?

  6. Write summary(event: Event): String, with a line of text for each kind of event. Leave one case out first and see what the compiler says.

9. Errors as values

Several methods in Fully Booked answer None when they can’t do what they were asked. With next and reserve, that leaves the caller guessing at the reason. A Java method in that position would probably throw an exception with a message in it. Scala code in the functional style returns the failure to its caller as an ordinary value, with a type that says what can go wrong.

9.1. Naming what can go wrong

Create errors.scala, with a type for the failures and a method that puts them into words:

package fullybooked

import java.time.LocalTime

enum BookingError:
  case NotAllowed(status: Status, event: Event)
  case PartyTooLarge(size: Int)
  case NothingFree(size: Int, time: LocalTime)

def explain(error: BookingError): String = error match
  case BookingError.NotAllowed(status, event) =>
    s"$event isn't possible when the booking is ${describe(status)}"
  case BookingError.PartyTooLarge(size) =>
    s"we can't seat $size, our largest table takes ${Booking.largestParty}"
  case BookingError.NothingFree(size, time) =>
    s"no table for $size is free at $time"

The errors are cases of an enum like Status. They aren’t exceptions: they have no stack trace and nothing throws them. NotAllowed holds the status and the event that didn’t go together. PartyTooLarge holds the size that was asked for, and NothingFree the size and the time. The wording is all in explain, where the compiler will warn us if we add an error and leave it out.

9.2. Either

Replace next in events.scala with a version that says why it refused:

def next(status: Status, event: Event): Either[BookingError, Status] = (status, event) match
  case (Status.Requested, Confirm)      => Right(Status.Confirmed)
  case (Status.Confirmed, Arrive(at))   => Right(Status.Seated(at))
  case (Status.Confirmed, FailToArrive) => Right(Status.NoShow)
  case (Status.Seated(_), Leave)        => Right(Status.Completed)
  case (Status.Requested | Status.Confirmed, Cancel(reason)) =>
    Right(Status.Cancelled(reason))
  case _ => Left(BookingError.NotAllowed(status, event))

The return type is now Either[BookingError, Status]. An Either is a Left or a Right. Here a Left holds a BookingError and a Right holds a Status. Nothing in the language says which side is the failure. The convention, which the standard library follows throughout, is that Right is the answer you wanted and Left is the error, with the error’s type written first.

The project won’t compile until handle and lifecycle have caught up with that. In model.scala, handle only needs its return type changed:

  def handle(event: Event): Either[BookingError, Booking] =
    next(status, event).map(newStatus => copy(status = newStatus))

On an Either, map changes the value inside a Right and passes a Left along untouched, so a refusal from next comes out of handle unchanged. flatMap and for skip a Left too, as they skipped None on an Option.

Replace lifecycle in main.scala with a version that matches on the two sides:

@main def lifecycle(): Unit =
  var booking = Booking(Party("Moreau", 3), Table(3, 4), LocalTime.of(20, 15))
  val events = List(Confirm, Arrive(LocalTime.of(20, 10)), Cancel("too noisy"), Leave)
  for event <- events do
    booking.handle(event) match
      case Right(changed) =>
        booking = changed
        println(s"$event: ${describe(booking.status)}")
      case Left(error) =>
        println(explain(error))
$ scala run . --main-class fullybooked.lifecycle
Confirm: confirmed
Arrive(20:10): at the table since 20:10
Cancel(too noisy) isn't possible when the booking is at the table since 20:10
Leave: been and gone

The third line now comes from explain, by way of the error that next returned. The tests in events.test.scala need bringing up to date as well. Here’s the whole file:

package fullybooked

import java.time.LocalTime

class EventTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)
  val requested = Booking(Party("Moreau", 3), Table(3, 4), eight)

  test("a booking goes from requested to completed"):
    val events = List(Confirm, Arrive(eight), Leave)
    val start: Either[BookingError, Booking] = Right(requested)
    val end = events.foldLeft(start): (booking, event) =>
      booking.flatMap(_.handle(event))
    assertEquals(end.map(_.status), Right(Status.Completed))

  test("a cancelled booking can't be seated"):
    val cancelled = requested.handle(Cancel("flu"))
    assertEquals(cancelled.map(_.status), Right(Status.Cancelled("flu")))
    assertEquals(
      cancelled.flatMap(_.handle(Arrive(eight))),
      Left(BookingError.NotAllowed(Status.Cancelled("flu"), Arrive(eight)))
    )

  test("a party at its table can't cancel"):
    val seated = requested.copy(status = Status.Seated(eight))
    assert(seated.handle(Cancel("changed our minds")).isLeft)

  test("seated parties hold their table"):
    assert(holdsTable(requested.copy(status = Status.Seated(eight))))

In the first test, start has its type written out. Left to itself the compiler would give a bare Right(requested) the type Right, and then reject the fold because handle can return a Left. Chapter 8 wrote Option(requested) to get round the same problem. The second test checks which error came back and what it holds. The third only cares that something failed, so it checks isLeft, which is true for any Left.

9.3. From an Option to an Either

In chapter 5 a party of five and a party of nine both got None from the restaurant, for different reasons. Replace reserve in availability.scala with a version that gives each its own error:

def reserve(store: BookingStore, party: Party, time: LocalTime): Either[BookingError, Booking] =
  if party.size > Booking.largestParty then Left(BookingError.PartyTooLarge(party.size))
  else
    bestTable(party, time, store.all)
      .toRight(BookingError.NothingFree(party.size, time))
      .map: table =>
        Thread.sleep(300)
        val booking = Booking(party, table, time, Status.Confirmed)
        store.add(booking)
        booking

A party bigger than the largest table gets a Left straight away. For the rest, bestTable still returns an Option[Table]. toRight turns that into an Either: a Some becomes a Right, and None becomes a Left holding the error we give it. I’ve left bestTable as it was. reserve turns the over-large party away before it calls bestTable, so a None from there can now only mean that nothing is free.

In desk.scala, replace reply so that it takes the new type and gets its wording from explain:

def reply(name: String, result: Either[BookingError, Booking]): String = result match
  case Right(booking) => s"$name: table ${booking.table.number} at ${booking.time}"
  case Left(error)    => s"$name: sorry, ${explain(error)}"

The project won’t compile again until lastTable and one test have caught up. In lastTable, replace the lines from var toIbrahim down to the two printlns that follow the joins once more:

  def ask(party: Party): String = reply(party.name, reserve(store, party, eight))
  var toIbrahim = ""
  var toCastellanos = ""
  val first = Thread(() => toIbrahim = ask(Party("Ibrahim", 7)))
  val second = Thread(() => toCastellanos = ask(Party("Castellanos", 8)))
  first.start()
  Thread.sleep(100)
  second.start()
  first.join()
  second.join()

  println(toIbrahim)
  println(toCastellanos)

The first new line defines ask inside lastTable. Scala allows a method inside a method. The inner one can use store and eight from the method around it. In store.test.scala, the test of reserve should now expect Right(Table(1, 2)) where it had Some(Table(1, 2)). Run the tests.

9.4. A request from outside

So far we’ve written every Party into the code ourselves. Real bookings arrive as text, from a web form or a command line, and text can be wrong in ways our types don’t allow inside the program: a size of “two”, a time of “8pm”. The place where outside text is turned into values of our own types is usually called the boundary. We’ll do all the checking there, once, so that everything behind it can rely on the types.

A request can fail in four more ways. Add the cases to the end of BookingError:

  case NameMissing
  case BadPartySize(text: String)
  case BadTime(text: String)
  case BadPhone(text: String)

Add their wording to the end of explain:

  case BookingError.NameMissing        => "we need a name for the booking"
  case BookingError.BadPartySize(text) => s"'$text' isn't a number of people"
  case BookingError.BadTime(text)      => s"'$text' isn't a time, try 19:30"
  case BookingError.BadPhone(text)     => s"'$text' isn't a phone number"

Now create request.scala:

package fullybooked

import java.time.LocalTime

case class BookingRequest(name: String, size: String, time: String, phone: String)

def nameIn(request: BookingRequest): Either[BookingError, String] =
  val name = request.name.trim
  if name.isEmpty then Left(BookingError.NameMissing) else Right(name)

def sizeIn(request: BookingRequest): Either[BookingError, Int] =
  request.size.toIntOption
    .filter(_ > 0)
    .toRight(BookingError.BadPartySize(request.size))

def phoneIn(request: BookingRequest): Either[BookingError, Option[Phone]] =
  if request.phone.isBlank then Right(None)
  else
    Phone.parse(request.phone) match
      case Some(phone) => Right(Some(phone))
      case None        => Left(BookingError.BadPhone(request.phone))

A BookingRequest is all strings, because that’s what arrives. Each of the three methods looks at one field and returns either a value of the type we really want or the reason it couldn’t produce one. In sizeIn, toIntOption gives None for “two”. filter then keeps a Some only when its number is above zero, so “0” and “-3” end up as None too. toRight turns the result into an Either. The trim and isBlank in the other two are methods on Java’s String.

The type of phoneIn has an Option inside an Either. No number at all is fine and gives Right(None). A good number gives Right(Some(phone)). Something that was typed but isn’t a number is a Left, so the customer from the end of chapter 8 can be told their number wasn’t understood.

9.5. Exceptions at the boundary

The time is harder, because the obvious way to parse it is LocalTime.parse, which throws a DateTimeParseException for “8pm”. Add import scala.util.Try under the other import in request.scala, then add this to the end of the file:

def timeIn(request: BookingRequest): Either[BookingError, LocalTime] =
  Try(LocalTime.parse(request.time)).toOption
    .toRight(BookingError.BadTime(request.time))

Wrapping the call in Try(...) runs it and catches whatever it throws, short of something fatal like an OutOfMemoryError. The result is a Success holding the value or a Failure holding the exception. toOption reduces that to a Some or a None, ready for toRight.

I use Try where our code calls a Java method that throws, and convert the result on the same line. Chapter 2 mentioned that Scala has try and catch, and exercise 2 has you write timeIn with them. I don’t pass a Try around, because the error inside one is always a Throwable. A caller learns no more from that than from throws Exception in Java.

I return an Either for failures that are part of the job, such as bad input, a full restaurant or a move that isn’t allowed. The caller is expected to deal with those, and the return type tells it they can happen. I don’t catch the exception from a bug or a lost database connection. Nobody near the failing line can do much about one of those, so it travels up to the top of the program.

9.6. The first problem or all of them

Add two methods to the end of request.scala. The first stops at the first check that fails, and the second runs all four:

def validateFast(request: BookingRequest): Either[BookingError, (Party, LocalTime)] =
  for
    name <- nameIn(request)
    size <- sizeIn(request)
    time <- timeIn(request)
    phone <- phoneIn(request)
  yield (Party(name, size, phone), time)

def validate(request: BookingRequest): Either[List[BookingError], (Party, LocalTime)] =
  (nameIn(request), sizeIn(request), timeIn(request), phoneIn(request)) match
    case (Right(name), Right(size), Right(time), Right(phone)) =>
      Right((Party(name, size, phone), time))
    case (name, size, time, phone) =>
      Left(List(name, size, time, phone).flatMap(_.left.toOption))

In validateFast the for runs on Either. Each <- takes the value out of a Right. At the first Left the for stops with that Left as its result. The later checks don’t run, so the caller hears about one problem at a time.

The other method, validate, has List[BookingError] as its error type. It runs all four checks and then matches on the results together. If every one is a Right, it builds the party. If not, left.toOption gives Some of the error for each Left and None for each Right, and flatMap keeps what’s inside the Somes.

Someone filling in a form would usually rather see every mistake at once, so I collect the errors when I’m validating input. Steps that depend on each other have to fail fast: there’s no point looking for a table for a party we couldn’t parse. The match on four results is long-winded. Chapter 11 shortens it with a library called Cats.

Create request.test.scala and run the tests:

package fullybooked

import java.time.LocalTime

class RequestTests extends munit.FunSuite:
  val good = BookingRequest("Silva", "2", "20:00", "07700 900199")

  test("a good request becomes a party and a time"):
    val expected = (Party("Silva", 2, Phone.parse("07700 900199")), LocalTime.of(20, 0))
    assertEquals(validate(good), Right(expected))

  test("the phone number can be left out"):
    assertEquals(validate(good.copy(phone = "")).map((party, _) => party.phone), Right(None))

  test("failing fast reports the first problem only"):
    val bad = good.copy(size = "two", time = "8pm")
    assertEquals(validateFast(bad), Left(BookingError.BadPartySize("two")))

  test("validate reports every problem"):
    val bad = good.copy(size = "two", time = "8pm")
    val expected = List(BookingError.BadPartySize("two"), BookingError.BadTime("8pm"))
    assertEquals(validate(bad), Left(expected))

9.7. Booking from the command line

Add one more @main to the end of main.scala, which takes its four arguments as text:

@main def book(name: String, size: String, time: String, phone: String): Unit =
  val store = InMemoryBookingStore()
  for booking <- tonight do store.add(booking)

  val request = BookingRequest(name, size, time, phone)
  val result = validate(request).flatMap: (party, time) =>
    reserve(store, party, time).left.map(error => List(error))

  result match
    case Right(booking) => println(label(booking))
    case Left(errors) =>
      println("Sorry, we couldn't make that booking:")
      for error <- errors do println(s"  ${explain(error)}")

Here validate fails with a list of errors and reserve with a single one. To give the chain one error type, left.map wraps the error from reserve in a list. It’s map for the other side: it changes a Left and leaves a Right alone. Inside the lambda, time is the parsed LocalTime and hides the String parameter of the same name.

$ scala run . --main-class fullybooked.book -- Silva 2 20:00 "07700 900199"
20:00 table 1: Silva, party of 2
$ scala run . --main-class fullybooked.book -- Silva two 8pm "ask for Dev"
Sorry, we couldn't make that booking:
  'two' isn't a number of people
  '8pm' isn't a time, try 19:30
  'ask for Dev' isn't a phone number
$ scala run . --main-class fullybooked.book -- Zielinski 9 20:00 ""
Sorry, we couldn't make that booking:
  we can't seat 9, our largest table takes 8
$ scala run . --main-class fullybooked.book -- Achterberg 5 20:00 ""
Sorry, we couldn't make that booking:
  no table for 5 is free at 20:00

The last two are the parties of nine and five from chapter 5.

9.8. Exercises

Solutions are in the book’s downloadable code, under ch09/exercises.

  1. Add an error KitchenClosed(time: LocalTime) and make reserve return it for a time before opens or after lastOrders, the two values from chapter 7. Test it.

  2. Write timeWithCatch, a version of timeIn that uses try and catch and no Try. The syntax is try ... catch case error: DateTimeParseException => ..., and both halves are expressions. You’ll need to import the exception from java.time.format.

  3. Predict each result and then check it in a test, starting from val four: Either[String, Int] = Right(4) and val no: Either[String, Int] = Left("no"): four.map(_ + 1), no.map(_ + 1), no.orElse(Right(1)) and four.toOption.

  4. Write seatAll(bookings: List[Booking], at: LocalTime): Either[BookingError, List[Booking]], which sends Arrive(at) to every booking and stops at the first that refuses. A foldLeft will do it, with a for inside the lambda and a starting value declared as an Either[BookingError, List[Booking]].

10. Generics and variance

When the restaurant is full the staff take a name and put the party on a waiting list. A waiting-list entry has a party and a time, as a booking does. The staff ask the same thing of both: who’s first? To write that question once, so that it works for bookings and for the waiting list, we need type parameters, which Java calls generics. Scala writes them in square brackets. It also has its own answer to whether a list of bookings can be used as a list of something more general.

10.1. One method for two types

Give the two types something in common. In model.scala, add a trait and a case class above Booking:

trait Timed:
  def time: LocalTime

case class WaitlistEntry(party: Party, time: LocalTime) extends Timed

Then make Booking extend the trait, by changing the line that closes its parameter list:

case class Booking(
    party: Party,
    table: Table,
    time: LocalTime,
    status: Status = Status.Requested
) extends Timed:

A field is allowed to stand in for a method that a trait declares. Each class already has a field called time, so neither needs anything added.

Create waitlist.scala with a first attempt at the question:

package fullybooked

import java.time.LocalTime

def earliestOf(items: List[Timed]): Option[Timed] =
  items.minByOption(_.time)

A call such as earliestOf(tonight) compiles and finds the Haddad booking, but its type is Option[Timed]. The compiler no longer knows there’s a booking inside, so earliestOf(tonight).map(_.party) is an error: a Timed has no party.

10.2. Type parameters and bounds

Add a second version to waitlist.scala:

def earliest[A <: Timed](items: List[A]): Option[A] =
  items.minByOption(_.time)

The A in square brackets is a type parameter. Java would write this method as <A extends Timed> Optional<A> earliest(List<A> items). Scala puts type parameters after the method’s name and spells extends as <:. Here A <: Timed gives A an upper bound: it can be Timed or any subtype of Timed. The method needs the bound because it uses time. Without one, A could be any type at all and the body wouldn’t compile.

Hand earliest a list of bookings and A is Booking, so an Option[Booking] comes back where earliestOf gave an Option[Timed]. The caller doesn’t have to say what A is, because the compiler infers it from the argument.

Create generics.test.scala:

package fullybooked

import java.time.LocalTime

class GenericsTests extends munit.FunSuite:
  val seven = Booking(Party("Okonkwo", 5), Table(5, 6), LocalTime.of(19, 30))
  val eight = Booking(Party("Lindqvist", 7), Table(6, 8), LocalTime.of(20, 0))
  val waiting = WaitlistEntry(Party("Silva", 2), LocalTime.of(19, 0))

  test("earliest keeps the type it was given"):
    assertEquals(earliest(List(eight, seven)).map(_.party.name), Some("Okonkwo"))
    assertEquals(earliest(List(waiting)).map(_.party.size), Some(2))

  test("a list of bookings is a list of timed things"):
    val bookings: List[Booking] = List(eight, seven)
    assertEquals(earliestOf(bookings).map(_.time), Some(LocalTime.of(19, 30)))
    assertEquals(earliestOf(List(eight, waiting)), Some(waiting))

Run the tests. The first reaches a party through the result of earliest, where earliestOf(tonight).map(_.party) was an error.

As in Java, the compiler erases type arguments, so a List[Booking] and a List[Table] are the same class once the program is running.

10.3. Is a list of bookings a list of timed things?

The second test passes bookings, a List[Booking], to earliestOf, whose signature asks for a List[Timed], and the compiler accepts it. In Java it wouldn’t, because a List<Booking> isn’t a List<Timed>. A Java method that wants to accept either has to ask for a List<? extends Timed>.

Scala lets the author of a class settle this once, where Java leaves it to every method that uses the class. The library declares its list as List[+A]. The plus sign means that a list of some type is also a list of each of that type’s supertypes, so a List[Booking] is a List[Timed]. A type parameter marked with a plus is covariant.

The plus is safe on List because a list can’t be changed. All anyone can do with a List[Timed] is take timed things out, and bookings are timed things. It wouldn’t be safe for a collection you can add to. Put import scala.collection.mutable.ArrayBuffer under the other import in waitlist.scala and add a method that takes a buffer, with a @main to call it:

def addLatecomer(arrivals: ArrayBuffer[Timed], time: LocalTime): Unit =
  arrivals += WaitlistEntry(Party("Latecomer", 2), time)

@main def latecomers(): Unit =
  val arrivals = ArrayBuffer(tonight(3), tonight(4))
  addLatecomer(arrivals, LocalTime.of(19, 45))
  println(arrivals.map(_.time))
$ scala compile .
[error] ./waitlist.scala:17:16
[error] Found:    (arrivals : scala.collection.mutable.ArrayBuffer[fullybooked.Booking])
[error] Required: scala.collection.mutable.ArrayBuffer[fullybooked.Timed]
[error]   addLatecomer(arrivals, LocalTime.of(19, 45))
[error]                ^^^^^^^^
Compilation failed

The compiler inferred ArrayBuffer[Booking] for arrivals from its two elements. If the call were allowed, addLatecomer would add a WaitlistEntry to it. The next line of code that took an element out expecting a Booking would then get something else. Java’s arrays do allow this kind of substitution and throw an ArrayStoreException at run time when it goes wrong. ArrayBuffer is declared with no sign on its type parameter, which makes it invariant: an ArrayBuffer[Booking] can’t be used as a buffer of anything else.

The fix is to say from the start that the buffer holds timed things, by giving the type argument in square brackets. Change the line that builds the buffer:

@main def latecomers(): Unit =
  val arrivals = ArrayBuffer[Timed](tonight(3), tonight(4))
  addLatecomer(arrivals, LocalTime.of(19, 45))
  println(arrivals.map(_.time))
$ scala run . --main-class fullybooked.latecomers
ArrayBuffer(19:30, 19:30, 19:45)

10.4. Functions go the other way

Add two more methods to the end of waitlist.scala, each of which takes a function for turning an item into text:

def timesOf(items: List[Timed], show: Timed => String): List[String] =
  items.map(show)

def bookingLines(bookings: List[Booking], show: Booking => String): List[String] =
  bookings.map(show)

Add a test to the end of GenericsTests:

  test("a function of any timed thing will do for bookings"):
    val clock: Timed => String = item => s"at ${item.time}"
    assertEquals(bookingLines(List(seven), clock), List("at 19:30"))

The test gives bookingLines a Timed => String where it asks for a Booking => String. That compiles, and it should: a function that can describe any timed thing can certainly describe a booking. timesOf couldn’t take a Booking => String, because it might hand that function a waiting-list entry.

So a function that takes something more general can be used in place of one that takes something more specific, the reverse of the rule for lists. The library declares one-argument functions as Function1[-T1, +R], with a minus on what goes in and a plus on what comes out. A type parameter marked with a minus is contravariant. The signature of map on a Java stream writes the same rule out at the point of use: Function<? super T, ? extends R>. A plus is allowed on a type parameter whose values only come out and a minus on one whose values only go in. A parameter used both ways gets neither.

I’d expect you to write very few of these signs and to read a great many. They’re all over the documentation of Scala libraries, and being able to read [-T1, +R] as “takes a T1, gives back an R” is most of what’s needed.

10.5. A type parameter of our own

The owner wants to see how each deposit was worked out as well as the figure. That calls for a value which carries notes along with it, whatever type the value happens to be. Create noted.scala:

package fullybooked

case class Noted[+A](value: A, notes: List[String]):
  def map[B](change: A => B): Noted[B] =
    Noted(change(value), notes)

  def flatMap[B](next: A => Noted[B]): Noted[B] =
    val result = next(value)
    Noted(result.value, notes ++ result.notes)

A class can take a type parameter as a method can. The plus on this one makes Noted covariant, which is safe because its value can only be read. Both methods change the value into some other type B. With map the notes stay as they are. The step passed to flatMap returns notes of its own, which go after the ones we already have.

Working out a deposit goes through three methods here, each returning an amount with any note it has to add. Add them to noted.scala with a fourth that chains them:

def charge(party: Party, name: String, rule: DepositRule): Noted[BigDecimal] =
  Noted(rule(party), List(s"$name rule for ${party.size} people: ${rule(party)}"))

def limit(amount: BigDecimal, most: BigDecimal): Noted[BigDecimal] =
  if amount > most then Noted(most, List(s"limited to $most")) else Noted(amount, List())

def regular(amount: BigDecimal, party: Party): Noted[BigDecimal] =
  if party.name == "Okonkwo" then Noted(amount / 2, List("half price for a regular"))
  else Noted(amount, List())

def quoteLonghand(party: Party): Noted[BigDecimal] =
  charge(party, "weekend", weekend).flatMap: full =>
    limit(full, BigDecimal(100)).flatMap: capped =>
      regular(capped, party).map: owed =>
        owed

The chain in quoteLonghand works, but each step is nested inside the one before and the last line just hands back owed. In a for, the three steps would sit one under another.

10.6. What for turns into

Chapter 4 said the compiler turns a for into calls to the collection methods. Start a REPL on the project, import fullybooked.* and compare:

scala> for booking <- tonight if booking.party.size > 4 yield booking.party.name
val res0: List[String] = List("Haddad", "Okonkwo", "Lindqvist")

scala> tonight.withFilter(booking => booking.party.size > 4).map(booking => booking.party.name)
val res1: List[String] = List("Haddad", "Okonkwo", "Lindqvist")

scala> for table <- Table.all.take(2); hour <- List(18, 19) yield (table.number, hour)
val res2: List[(Int, Int)] = List((1, 18), (1, 19), (2, 18), (2, 19))

scala> Table.all.take(2).flatMap(table => List(18, 19).map(hour => (table.number, hour)))
val res3: List[(Int, Int)] = List((1, 18), (1, 19), (2, 18), (2, 19))

Each pair gives the same answer because the second line of the pair is what the compiler makes of the first. In the first pair the only <- became a map and the if became a call to withFilter in front of it. That method does what filter does without building a list in between. The second pair has two <- clauses. The last is a map again and each one before it is a flatMap wrapped around what follows, nested as quoteLonghand is. A for that ends in do has no yield and becomes a call to foreach, which runs a function on each element for its side effects.

The translation goes by method names alone and doesn’t care whether the thing on the right of the <- is a collection. That’s why for worked on Option in chapter 5 and on Either in chapter 9. Noted has a map and a flatMap, so a for will work on it too. Add this to noted.scala:

def quoteFor(party: Party): Noted[BigDecimal] =
  for
    full <- charge(party, "weekend", weekend)
    capped <- limit(full, BigDecimal(100))
    owed <- regular(capped, party)
  yield owed

Add a @main to the end of main.scala:

@main def quote(name: String, size: Int): Unit =
  val result = quoteFor(Party(name, size))
  println(s"Deposit for $name: ${result.value}")
  for note <- result.notes do println(s"  $note")
$ scala run . --main-class fullybooked.quote -- Okonkwo 8
Deposit for Okonkwo: 50
  weekend rule for 8 people: 120
  limited to 100
  half price for a regular

None of the three steps in quoteFor mentions the notes, which flatMap collected along the way. Add a test to the end of GenericsTests to confirm that the two versions agree:

  test("a quote carries its notes"):
    val quote = quoteFor(Party("Okonkwo", 8))
    assertEquals(quote.value, BigDecimal(50))
    assertEquals(quote.notes.size, 3)
    assertEquals(quoteLonghand(Party("Okonkwo", 8)), quote)

10.7. A bound in the other direction

A replace method on Noted would swap the value for a new one and keep the notes. Written as def replace(other: A): Noted[A], it’s rejected by the compiler with “covariant type A occurs in contravariant position”. The plus sign promised that values of type A only come out, and here one goes in. If that were allowed, a Noted[Booking] being passed around as a Noted[Timed] could be handed a waiting-list entry, although its replace only promised to take bookings. The compiler goes by the signature and rejects it whatever the body would do.

Add this version inside Noted instead:

  def replace[B >: A](other: B): Noted[B] =
    Noted(other, notes)

The B >: A here is a lower bound: B is A or something above it. Replacing a booking with a waiting-list entry is fine as long as the result is no longer called a Noted[Booking]. The compiler picks B for itself: a val holding the result has the type Noted[Timed]. Add one last test to the end of GenericsTests:

  test("a noted booking is a noted timed thing"):
    val noted: Noted[Booking] = Noted(seven, List("moved from table 3"))
    val general: Noted[Timed] = noted
    assertEquals(general.notes, noted.notes)
    assertEquals(noted.replace(waiting).value.time, LocalTime.of(19, 0))

In the library, getOrElse on Option[A] is declared with a lower bound: getOrElse[B >: A](default: => B): B. That’s why lookUp went on compiling in chapter 8. Asked for the value of an Option[Phone] or else a String, the compiler chose a B above both, the union type Phone | String.

10.8. Exercises

Solutions are in the book’s downloadable code, under ch10/exercises.

  1. Write latest, the counterpart of earliest, and use it to find the name of tonight’s last party.

  2. Write pairs[A, B](first: List[A], second: List[B]): List[(A, B)], which gives every combination of one element from each list. It has two type parameters and no bounds.

  3. Rewrite validateFast from chapter 9 without for, as the compiler would.

  4. Write plain[A](value: A): Noted[A], which starts a value off with no notes, and withNote[A](noted: Noted[A], note: String): Noted[A], which adds one. Use flatMap for the second.

  5. Take the + out of Noted[+A] and compile the tests. Which line fails, and why does replace still compile?

  6. Decide which of these compile, then try them: val a: Option[Timed] = Some(tonight.head), val b: Booking => Timed = (item: Timed) => item and val c: Timed => Booking = (booking: Booking) => booking.

11. Givens and type classes

Start a REPL on the project with scala repl ., import fullybooked.* and java.time.LocalTime, and ask it to sort two things:

scala> tonight.map(_.time).sorted.take(3)
val res0: List[LocalTime] = List(18:00, 18:30, 18:45)

scala> tonight.sorted
-- [E172] Type Error: ------------------------------------------------------------------------------
1 |tonight.sorted
  |              ^
  |No given instance of type Ordering[fullybooked.Booking] was found for parameter ord of method sorted in trait StrictOptimizedSeqOps.
  |I found:
  |
  |    scala.math.Ordering.ordered[fullybooked.Booking](
  |      /* missing */summon[scala.math.Ordering.AsComparable[fullybooked.Booking]])
  |
  |But no implicit values were found that match type scala.math.Ordering.AsComparable[fullybooked.Booking].
  |
  |The following import might make progress towards fixing the problem:
  |
  |  import scala.math.Ordered.orderingToOrdered
  |
1 error found

The times sort and the bookings don’t. The error says what’s missing: “No given instance of type Ordering[fullybooked.Booking] was found for parameter ord of method sorted”. So sorted has a parameter we’ve never passed, an Ordering[Booking] called ord. The compiler fills in a parameter like that one by looking for a value of the right type.

11.1. A value the compiler can find

An Ordering[A] is Scala’s Comparator<A>, an object that knows how to compare two values of type A. The library has one for LocalTime because LocalTime is Comparable. It has none for Booking and can’t guess whether we’d want bookings in order of time, name or size.

Add a line to the Booking companion in model.scala, above walkIn, to supply one:

  given Ordering[Booking] = Ordering.by(booking => booking.time)

The word given declares a value that the compiler is allowed to pass on our behalf. It needs no name, because it’s looked up by type. Ordering.by builds an ordering from a function, here the one that picks out a booking’s time. Restart the REPL and import fullybooked.* again:

scala> tonight.sorted.map(_.party.name).take(3)
val res0: List[String] = List("Haddad", "Nakamura", "Abara")

scala> tonight.min.party.name
val res1: String = "Haddad"

scala> tonight.max.party.name
val res2: String = "Fitzgerald"

scala> tonight.sorted(using Ordering.by(booking => booking.party.size)).map(_.party.size)
val res3: List[Int] = List(2, 2, 2, 3, 4, 5, 6, 7)

All three of sorted, min and max work on bookings now, though none of the calls mentions the ordering. max gives the booking with the latest time. The last line passes a different ordering by hand, after the keyword using, to sort by party size. Any call can override the given that way.

When a method wants an Ordering[Booking], the compiler looks in two places. One is the code around the call: any given declared there or imported into it. Givens need their own form of import, import some.pkg.given, because an import ending in .* leaves them out. The other place is the companion of each type involved, which is why an ordering in the companion of Booking is available everywhere without an import.

11.2. A type class of our own

Fully Booked has collected several methods that turn a value into text for the staff: describe for a status, explain for an error, label for a booking. One method that prints a list of statuses, errors or bookings would have to know which of those names goes with the type it was given. Create display.scala:

package fullybooked

trait Display[A]:
  def text(value: A): String

object Display:
  given Display[Status]:
    def text(status: Status): String = describe(status)

  given Display[BookingError]:
    def text(error: BookingError): String = explain(error)

  given Display[Booking]:
    def text(booking: Booking): String = label(booking)

The trait Display[A] has one method, text, which turns an A into a string. The three givens in the companion each implement the trait for one type, using the method we already had. The colon after given Display[Status] opens a body, like the colon after a class header.

To get this in Java we’d have Status implement an interface. Our Status hasn’t been touched and doesn’t know that Display exists, so a Display[LocalTime] for a class we don’t own would be just as easy to write. A trait with a type parameter and a set of given instances, as Display and Ordering both are, is called a type class.

Now a method can ask for any type that has a Display. Add these to display.scala:

def lines[A](items: List[A])(using display: Display[A]): List[String] =
  items.map(item => display.text(item))

def report[A: Display](title: String, items: List[A]): String =
  (title :: lines(items).map(line => s"  $line")).mkString("\n")

The second parameter list of lines starts with using. Callers leave it out and the compiler fills it in with the given for whatever A turns out to be. If there’s no such given, the call doesn’t compile.

In report, [A: Display] is a context bound: it means “any A that has a given Display”, without giving that value a name. report doesn’t need the name, because it only hands the Display on to lines and the compiler does that as well. The mkString at the end joins a list of strings into one, with its argument between them.

11.3. Extension methods

It would be easier to write item.shown than display.text(item). Scala can add a method to a type from the outside. Add this to the end of display.scala:

extension [A](value: A)(using display: Display[A])
  def shown: String = display.text(value)

An extension names the value it extends, in parentheses. Its methods are then written as though they were inside that value’s class. The compiler turns value.shown into an ordinary call with value as its argument, the job a static helper method does in Java. This one applies to any type A that has a Display, so booking.shown and status.shown both work.

A Phone has had no methods of its own since it became an opaque type in chapter 8. An extension inside phone.scala can see that a Phone is a String, and so can use take for the first five characters and drop for the rest. Add this to the end of phone.scala:

extension (phone: Phone)
  def formatted: String = s"${phone.take(5)} ${phone.drop(5)}"

Add a test to the end of PhoneTests:

  test("a number is formatted with its space"):
    assertEquals(Phone.parse("07700900141").map(_.formatted), Some("07700 900141"))

The .only we put after a test’s name in chapter 6 is a method that String doesn’t have. That looks like an extension method, but MUnit was written for Scala 2 as well as Scala 3 and uses Scala 2’s mechanism, an implicit conversion. The conversion turns the string into an object of MUnit’s own that does have only. Scala 2 used the one keyword implicit for conversions, for what are now givens and for using parameters, which is why this chapter’s first error message talked about “implicit values”. Appendix A covers reading that older style.

Add a new @main to the end of main.scala that uses the ordering, report and shown. It calls distinct on the statuses to remove the repeats from that list:

@main def nightReport(): Unit =
  println(report("Tonight", tonight.sorted))
  println(report("Statuses", tonight.map(_.status).distinct))
  println(s"First in: ${tonight.min.shown}")
$ scala run . --main-class fullybooked.nightReport
Tonight
  18:00 table 5: Haddad, party of 6
  18:30 table 2: Nakamura, party of 2
  18:45 table 4: Abara, party of 4
  19:30 table 5: Okonkwo, party of 5
  19:30 table 1: Patel, party of 2
  20:00 table 6: Lindqvist, party of 7 (deposit taken)
  20:15 table 3: Moreau, party of 3
  21:15 table 2: Fitzgerald, party of 2
Statuses
  cancelled (rang to cancel)
  at the table since 18:25
  at the table since 18:50
  confirmed
  waiting to be confirmed
First in: 18:00 table 5: Haddad, party of 6

11.4. Instances the compiler writes

For some type classes the compiler can write the instance itself, if you ask with derives. Try it in a plain scala repl:

scala> case class Seat(number: Int)
// defined case class Seat

scala> case class Chair(number: Int)
// defined case class Chair

scala> Seat(4) == Chair(4)
val res0: Boolean = false

scala> import scala.language.strictEquality

scala> Seat(4) == Chair(4)
-- [E172] Type Error: ------------------------------------------------------------------------------
1 |Seat(4) == Chair(4)
  |^^^^^^^^^^^^^^^^^^^
  |Values of types Seat and Chair cannot be compared with == or !=
1 error found

scala> Seat(4) == Seat(4)
-- [E172] Type Error: ------------------------------------------------------------------------------
1 |Seat(4) == Seat(4)
  |^^^^^^^^^^^^^^^^^^
  |Values of types Seat and Seat cannot be compared with == or !=
1 error found

scala> case class Stool(legs: Int) derives CanEqual
// defined case class Stool

scala> Stool(3) == Stool(3)
val res3: Boolean = true

By default Scala will compare a Seat with a Chair, two classes that have nothing to do with each other, and the answer is just false. A comparison like that is nearly always a mistake. The import switches on a stricter rule, under which two types can only be compared if there’s a CanEqual for them. Under that rule a Seat can’t even be compared with another Seat. A Stool can, because derives CanEqual asks the compiler to write the instance for it. I haven’t turned strict equality on for Fully Booked, because Status, BookingError and Event would then each need derives CanEqual and a Java type such as LocalTime would need a given written for it. Libraries use derives a great deal: chapter 13 uses it for a configuration reader and chapter 19 for JSON.

11.5. Cats

Cats is a library of type classes, with instances for the standard types and extension methods for using them. Most of the libraries in the rest of this book are built on it. It’s the first dependency that isn’t only for tests, so the line to add to project.scala, above the two for tests, says dep:

//> using dep org.typelevel::cats-core:2.13.0

Create summary.scala, which uses the type class Monoid:

package fullybooked

import cats.Monoid
import cats.syntax.all.*
import java.time.LocalTime

case class Summary(bookings: Int, covers: Int, deposits: BigDecimal)

object Summary:
  given Monoid[Summary]:
    def empty: Summary = Summary(0, 0, BigDecimal(0))
    def combine(a: Summary, b: Summary): Summary =
      Summary(a.bookings + b.bookings, a.covers + b.covers, a.deposits + b.deposits)

def summarise(bookings: List[Booking], rule: DepositRule): Summary =
  bookings
    .filter(holdsTable)
    .foldMap(booking => Summary(1, booking.party.size, rule(booking.party)))

A Monoid[A] has two things: a way to combine two values of A into one, and an empty value that changes nothing when it’s combined with another. Numbers under addition are a monoid, with 0 as the empty value. Strings, lists and maps have instances as well, and we’ve just given Summary one. foldMap, which import cats.syntax.all.* adds to List, turns each element into a value with a Monoid and combines them all, starting from empty. In chapter 7, depositsDue did that for one number with map and sum. foldMap does it for three figures at once and would do it for anything else we gave a Monoid.

Cats has a method, traverse, for the job that seatAll did with a fold and a for in chapter 9’s fourth exercise. If you kept your own seatAll, delete it, then add this to summary.scala:

def seatAll(bookings: List[Booking], at: LocalTime): Either[BookingError, List[Booking]] =
  bookings.traverse(booking => booking.handle(Arrive(at)))

Mapping handle over the list would give a List[Either[BookingError, Booking]], a list of separate results. traverse gives an Either[BookingError, List[Booking]]: every booking seated, or the first error. With a function that returns an Option it gives an Option of a list, which is None if any element gave None.

The match on four results in validate can go as well. In request.scala, add import cats.data.EitherNel and import cats.syntax.all.* to the imports and replace validate:

def validate(request: BookingRequest): EitherNel[BookingError, (Party, LocalTime)] =
  (
    nameIn(request).toEitherNel,
    sizeIn(request).toEitherNel,
    timeIn(request).toEitherNel,
    phoneIn(request).toEitherNel
  ).parMapN((name, size, time, phone) => (Party(name, size, phone), time))

The new validate still runs all four checks. parMapN takes the tuple of their results and a function of four arguments. It calls the function if every result is a Right and otherwise joins the errors together. Two BookingErrors can’t be joined into one, but two lists of them can, so toEitherNel puts each check’s error into a NonEmptyList, Cats’s list that can’t be empty. An Either with a NonEmptyList on the left is an EitherNel, the new return type. A list that can’t be empty is the accurate type for the errors of a failed validation, since there’s always at least one. I’ve used parMapN because Cats also has a mapN, which stops at the first error on an Either as a for would.

The project won’t compile until main.scala and one test have caught up with the new return type. In book, in main.scala, the line after validate becomes reserve(store, party, time).toEitherNel, and the loop that prints the errors becomes for error <- errors.toList do println(s" ${error.shown}"). A NonEmptyList has no foreach for the for to use, hence the toList. Add import cats.syntax.all.* under the package line of that file too. In request.test.scala, import cats.data.NonEmptyList and build the expected errors with NonEmptyList.of(...) where the test had List(...). The output for a bad request hasn’t changed:

$ scala run . --main-class fullybooked.book -- Silva two 8pm "ask for Dev"
Sorry, we couldn't make that booking:
  'two' isn't a number of people
  '8pm' isn't a time, try 19:30
  'ask for Dev' isn't a phone number

Finish with a test file, cats.test.scala, and run the tests:

package fullybooked

import cats.syntax.all.*
import java.time.LocalTime

class CatsTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)
  val confirmed = Booking(Party("Moreau", 3), Table(3, 4), eight, Status.Confirmed)

  test("maps of covers combine hour by hour"):
    val friday = Map(18 -> 6, 19 -> 5)
    val saturday = Map(19 -> 4, 20 -> 10)
    assertEquals(friday |+| saturday, Map(18 -> 6, 19 -> 9, 20 -> 10))

  test("a summary of tonight"):
    assertEquals(summarise(tonight, weekend), Summary(6, 23, BigDecimal(240)))
    assertEquals(summarise(List(), weekend), Summary(0, 0, BigDecimal(0)))

  test("seatAll seats everyone or reports the first booking that can't be"):
    val cancelled = confirmed.copy(status = Status.Cancelled("flu"))
    assertEquals(seatAll(List(confirmed, confirmed), eight).map(_.size), Right(2))
    assert(seatAll(List(confirmed, cancelled, confirmed), eight).isLeft)

The operator |+| is another way to write combine. On two maps it combines the values that share a key, so seven o’clock’s covers are added and the other hours are kept as they were.

11.6. Functor, monad and other names

Cats and its documentation use names from mathematics for things you’ve already been using. A type with a map, such as List, Option, Either or our own Noted, is a functor. A type that also has flatMap, along with some way of wrapping a plain value such as Some(x) or Right(x), is a monad. The type class that let parMapN join the error lists is called Semigroup, which is a Monoid without the empty.

Each of those is a type class in Cats, with a few laws that an instance is supposed to obey. One of the laws for a functor says that mapping with a function that returns its argument unchanged gives back an equal value. I’ve never needed the mathematics to use the type classes. When a library’s documentation says a type “is a monad”, I read it as “has flatMap, so it works in a for” and carry on.

11.7. Exercises

Solutions are in the book’s downloadable code, under ch11/exercises.

  1. Write a given Ordering[Table] that puts the largest tables first and breaks ties by table number. Ordering.by accepts a function that returns a tuple, and a negative number of seats sorts the large ones first.

  2. LocalTime is Java’s, but it can still be extended. Write an extension method isWithin(from: LocalTime, to: LocalTime): Boolean on it, and test it against opens and lastOrders.

  3. Write a Display[Table]. Then write one for a list of anything displayable, which starts given [A] => (display: Display[A]) => Display[List[A]]: and joins the items with commas.

  4. Write a case class Largest(size: Int) with a Monoid whose combine keeps the bigger of two, and use foldMap to find tonight’s largest party. What should empty be?

  5. Write parseAll(numbers: List[String]): Either[BookingError, List[Phone]] with traverse, giving BadPhone for the first text that isn’t a number.

Part III: Effects

12. Build your own IO

Chapter 1 said that functional Scala wraps a database query or an HTTP call in a type called IO, and that IO takes more getting used to than the rest of the style. The IO that services use comes from the Cats Effect library. We’ll write a small one of our own first. It takes about a dozen lines and has the methods we’ll use most on the real one.

12.1. A refactoring that changes the program

Create io.test.scala with one test:

package fullybooked

import java.time.LocalTime

class IOTests extends munit.FunSuite:
  val eight = LocalTime.of(20, 0)
  val silva = Party("Silva", 2)

  test("calling reserve twice is not the same as using its result twice"):
    val once = InMemoryBookingStore()
    val result = reserve(once, silva, eight)
    val pair = (result, result)
    assertEquals(once.all.size, 1)

    val again = InMemoryBookingStore()
    val other = (reserve(again, silva, eight), reserve(again, silva, eight))
    assertEquals(again.all.size, 2)

Both halves build a pair from a reservation for the Silvas. The first half makes the call once, gives the result a name and uses the name twice. The second writes the call out twice. If the expression were silva.size, naming it once or writing it twice would give the same program. With reserve the two are different programs: one store ends up with one booking and the other with two.

As well as working out a value, reserve changes the store. Chapter 4 used the term side effect for the printing a loop does. A change to the store is one too. Once a method has side effects, a call to it can’t be moved, named, repeated or dropped without asking what else would change. Before we can read a line that calls such a method, we have to know what the method goes on to do.

One side effect has been slowing the tests down since chapter 6. The Thread.sleep(300) inside reserve stands for a slow database, so every test that calls reserve waits for it. The delay belongs in a store, since it’s the store that plays the database. Delete that line from reserve in availability.scala and add a second store to store.scala that sleeps before it saves:

class SlowStore(delay: Long) extends InMemoryBookingStore:
  override def add(booking: Booking): Unit =
    Thread.sleep(delay)
    super.add(booking)

A SlowStore is an InMemoryBookingStore with a different add. Scala insists on the keyword override there, where Java’s @Override is optional. The new add calls the original through super.add. In lastTable, in main.scala, build the store with SlowStore(300) in place of InMemoryBookingStore(). lastTable still double-books table 6 and takes a second or two longer about it, because seating the regulars goes through the slow store too. No test uses the slow store, so the tests no longer wait.

12.2. A description in place of an action

Suppose reserve returned a description of the reservation to be made, and making it were a separate step. Then naming the description, or writing it twice, would change nothing, because nothing would have happened yet. Create io.scala, with a class for descriptions of that kind:

package fullybooked

class IO[+A](val unsafeRun: () => A):
  def map[B](change: A => B): IO[B] =
    IO(() => change(unsafeRun()))

  def flatMap[B](next: A => IO[B]): IO[B] =
    IO(() => next(unsafeRun()).unsafeRun())

object IO:
  def delay[A](body: => A): IO[A] = IO(() => body)
  def pure[A](value: A): IO[A] = IO(() => value)

def printLine(text: String): IO[Unit] = IO.delay(println(text))

An IO[A] holds a function of no arguments that produces an A. None of the work inside that function is done until someone calls unsafeRun. The plus makes IO covariant, which is safe because an A only ever comes out of it. In the companion, delay takes its parameter by name: it’s written => A, with nothing to the left of the arrow, as the parameter of whenOpen was in chapter 7. The code we hand to delay is stored in the function without being run. A value we already have can be wrapped with pure.

The signatures of map and flatMap are the ones Noted had in chapter 10. Each returns a new IO around a new function, so calling map runs nothing. It builds a bigger description, one that will run this IO and then apply change when somebody calls unsafeRun on it. printLine gives us the first useful one: a println that waits to be run. From here on I’ll call an IO a program, since a program is what it describes.

I’ve used Cats Effect’s names where I could, IO, delay and pure, so that chapter 13 has less to rename. Cats Effect’s method for running an IO is called unsafeRunSync.

Create evening.scala, with the reservation as an IO:

package fullybooked

import java.time.LocalTime

def reserveIO(
    store: BookingStore,
    party: Party,
    time: LocalTime
): IO[Either[BookingError, Booking]] =
  IO.delay(reserve(store, party, time))

And add two methods to the end of io.scala, each of which takes a program and returns a bigger one:

def twice[A](io: IO[A]): IO[(A, A)] =
  for
    first <- io
    second <- io
  yield (first, second)

def retry[E, A](attempt: IO[Either[E, A]], times: Int): IO[Either[E, A]] =
  attempt.flatMap:
    case Left(error) if times > 1 => retry(attempt, times - 1)
    case result                   => IO.pure(result)

The for in twice works because IO has a map and a flatMap. Each <- says that when the program runs, it will run the IO on the right of the arrow and give the result the name on the left. Add two tests to the end of IOTests:

  test("building an IO does nothing"):
    val store = InMemoryBookingStore()
    val booking = reserveIO(store, silva, eight)
    assertEquals(store.all.size, 0)
    booking.unsafeRun()
    assertEquals(store.all.size, 1)

  test("an IO can be named or written out twice, and it's the same program"):
    val named = InMemoryBookingStore()
    val booking = reserveIO(named, silva, eight)
    twice(booking).unsafeRun()
    assertEquals(named.all.size, 2)

    val written = InMemoryBookingStore()
    val both =
      for
        first <- reserveIO(written, silva, eight)
        second <- reserveIO(written, silva, eight)
      yield (first, second)
    both.unsafeRun()
    assertEquals(written.all.size, 2)

In the first test the store is still empty after the call to reserveIO, and the booking only appears when unsafeRun is called. The second repeats the chapter’s opening test with IO: one half names the reservation and passes the name to twice, and the other writes reserveIO out twice in a for. Both stores end up with two bookings. Run the tests and the three in IOTests pass.

12.3. A program is a value too

Add a bigger program to evening.scala:

def walkInIO(store: BookingStore, party: Party, time: LocalTime): IO[Unit] =
  for
    _ <- printLine(s"${party.name}, party of ${party.size}, asks for a table at $time")
    result <- reserveIO(store, party, time)
    _ <- printLine(reply(party.name, result))
  yield ()

@main def walkIns(): Unit =
  val store = InMemoryBookingStore()
  val eight = LocalTime.of(20, 0)
  val program =
    for
      _ <- walkInIO(store, Party("Silva", 2), eight)
      _ <- walkInIO(store, Party("Zielinski", 9), eight)
      count <- IO.delay(store.all.size)
      _ <- printLine(s"$count booking in the store")
    yield ()
  println("Nothing has happened yet")
  program.unsafeRun()

In walkInIO only the middle step has a result that’s needed, so the other two are written _ <-. program runs walkInIO for two parties, then counts the bookings and prints the count. By the time println runs, program exists, though nothing in it has run:

$ scala run . --main-class fullybooked.walkIns
Nothing has happened yet
Silva, party of 2, asks for a table at 20:00
Silva: table 1 at 20:00
Zielinski, party of 9, asks for a table at 20:00
Zielinski: sorry, we can't seat 9, our largest table takes 8
1 booking in the store

“Nothing has happened yet” is printed first, although its line comes after the whole of program in the source. Both walk-ins and the count are in that value. Nothing happens until the last line of walkIns calls program.unsafeRun(). A whole application can be written like this: the code builds one description, however large, and main runs it once. The place where it’s run is often called the edge of the program. The word unsafe in unsafeRun is there to make any other call to it look out of place.

12.4. What a value can do

The other method we added, retry, takes a program that might fail with a Left and returns a program that makes up to times attempts. Its lambda is written as two cases. The first has an if after its pattern, so it matches a Left only while more attempts are allowed, and it builds a retry with one attempt fewer. Any other result matches the second pattern, a bare name, and is handed back by IO.pure. Add a test for it to the end of IOTests:

  test("retry runs an IO again until it gives a Right"):
    var calls = 0
    val flaky: IO[Either[String, Int]] = IO.delay:
      calls = calls + 1
      if calls < 3 then Left("not yet") else Right(calls)
    assertEquals(retry(flaky, 5).unsafeRun(), Right(3))

In the test, flaky fails twice and then succeeds, so retry(flaky, 5) runs it three times. A retry for a plain method like reserve would need the call passed by name or wrapped in a function. An ordinary argument is worked out once, before the method starts. The result is all the method gets. What retry gets in attempt is the description, which it can run as many times as it needs.

12.5. What ours lacks

Add one more method to io.scala and one more test to IOTests:

def countdown(from: Int): IO[Unit] =
  if from == 0 then IO.pure(())
  else IO.delay(()).flatMap(_ => countdown(from - 1))
  test("a long chain of flatMaps runs out of stack"):
    countdown(1000).unsafeRun()
    val outcome =
      try
        countdown(1000000).unsafeRun()
        "finished"
      catch case error: StackOverflowError => "stack overflow"
    assertEquals(outcome, "stack overflow")

The new method, countdown, chains one flatMap for each number. A thousand is fine, but a million ends in a StackOverflowError. The test catches it with a try and catch, whose handler is written as a case. Look at flatMap to see why the stack overflows. Each step calls unsafeRun on the next from inside its own unsafeRun, so a program of a million steps needs a million nested calls. A loop that never ends, such as a server waiting for requests, would be a chain like that with no last step.

An exception thrown inside our IO flies straight out of unsafeRun, since the class has no method for handling it as a value. Everything runs on the calling thread, one step after another, and a program that’s been started can’t be cancelled or given a time limit. Nor does anything make sure that a file or a connection is closed afterwards.

Cats Effect’s IO deals with every one of those, and from the outside it still looks like ours: delay, pure, map, flatMap and for. Chapter 13 deletes io.scala and uses the real one in its place.

12.6. Exercises

Solutions are in the book’s downloadable code, under ch12/exercises.

  1. Write readLine: IO[String] around scala.io.StdIn.readLine(), and a program that asks for a name and then thanks the person by name. Run it from a @main.

  2. Write after[A, B](first: IO[A], second: IO[B]): IO[B], which runs the first program, ignores its result and runs the second.

  3. Write sequence[A](ios: List[IO[A]]): IO[List[A]], which turns a list of programs into one program that produces a list. Test that nothing runs until unsafeRun.

  4. Write timed[A](io: IO[A]): IO[(A, Long)], which gives a program’s result together with how many milliseconds it took. Reading the clock is a side effect as well, so System.nanoTime() belongs inside IO.delay.

  5. Write attempt[A](io: IO[A]): IO[Either[Throwable, A]], which turns an exception thrown while a program runs into a Left.

  6. countdown(1000000) overflows the stack when it’s run. Why doesn’t it overflow when it’s built?

13. Cats Effect

Delete io.scala and io.test.scala. The project stops compiling, because evening.scala has lost its IO. The replacement comes from Cats Effect. Libraries like it are known as effect systems: they supply an IO type and the runtime that runs it. The service in Part IV is built from http4s and doobie, which are both written on top of Cats Effect.

13.1. The real IO

In project.scala, add the dependency under the one for Cats:

//> using dep org.typelevel::cats-effect:3.7.1

Cats Effect is another library compiled with Scala 3.3, so the Unsafe warning that chapter 6 switched off for tests would now appear on every scala run. Change the last line of the file from test.javaOpt to javaOpt, which applies the option to both:

//> using javaOpt --sun-misc-unsafe-memory-access=allow

Then replace the whole of evening.scala:

package fullybooked

import cats.effect.{IO, IOApp}
import java.time.LocalTime

def reserveIO(
    store: BookingStore,
    party: Party,
    time: LocalTime
): IO[Either[BookingError, Booking]] =
  IO.blocking(reserve(store, party, time))

def walkInIO(store: BookingStore, party: Party, time: LocalTime): IO[Unit] =
  for
    _ <- IO.println(s"${party.name}, party of ${party.size}, asks for a table at $time")
    result <- reserveIO(store, party, time)
    _ <- IO.println(reply(party.name, result))
  yield ()

object WalkIns extends IOApp.Simple:
  val eight = LocalTime.of(20, 0)

  def run: IO[Unit] =
    for
      store <- IO.delay(InMemoryBookingStore())
      _ <- walkInIO(store, Party("Silva", 2), eight)
      _ <- walkInIO(store, Party("Zielinski", 9), eight)
      count <- IO.delay(store.all.size)
      _ <- IO.println(s"$count booking in the store")
    yield ()

Don’t leave out the import of cats.effect.IO, in this file or any later one. Java 25 has a small console class of its own called java.lang.IO. Without the import the compiler finds that one and reports that “IO does not take type parameters”.

Cats Effect has an IO.println, so our printLine has gone from walkInIO. reserveIO now uses IO.blocking where it had IO.delay. Both wrap code in an IO, and the section on threads below says how they differ.

The @main method has become WalkIns, an object that extends IOApp.Simple and supplies run, the program. IOApp provides the main method, starts the threads that Cats Effect runs on, runs our IO and shuts everything down afterwards. That main method is the edge of the program from chapter 12, and since the library wrote it, WalkIns calls nothing with unsafe in its name. Creating the store has moved inside the for as well. IO.delay(InMemoryBookingStore()) makes a new store each time the program runs, where a val outside the for would be made once and shared by every run.

An object is run by its name:

$ scala run . --main-class fullybooked.WalkIns
Silva, party of 2, asks for a table at 20:00
Silva: table 1 at 20:00
Zielinski, party of 9, asks for a table at 20:00
Zielinski: sorry, we can't seat 9, our largest table takes 8
1 booking in the store

13.2. When something throws

Create effects.test.scala:

package fullybooked

import cats.effect.{IO, Resource}
import cats.effect.unsafe.implicits.global

class EffectTests extends munit.FunSuite:
  val failing: IO[Int] = IO.delay("seven".toInt)

  test("an exception inside an IO is a value until someone asks for it"):
    val result = failing.attempt.unsafeRunSync()
    assert(result.isLeft)

  test("handleErrorWith swaps a failed program for another one"):
    assertEquals(failing.handleErrorWith(error => IO.pure(0)).unsafeRunSync(), 0)

A test has no IOApp, so it runs its programs by hand with unsafeRunSync(), until chapter 21 brings in a tidier way. That method takes the threads Cats Effect runs on as a using parameter, and the second import supplies them. Nothing uses the Resource in the first import until the next section. Run the tests.

Inside failing, "seven".toInt throws a NumberFormatException. Our own IO let an exception like that fly out of unsafeRun. The real one catches it and the program becomes a failed IO, in which every later step is skipped until something handles the failure. In the first test, attempt turns the IO[Int] into an IO[Either[Throwable, Int]] that never fails, with the exception as a Left. In the second test, handleErrorWith is given a function that takes the exception and returns a program to run instead of the failed one, which here is IO.pure(0).

When reserveIO meets a full restaurant, the result is a Left inside a successful IO. A write to a disk that’s full would make the IO itself fail. Chapter 9 split failures that way: the ones that are part of the job were returned as values and the rest were left as exceptions.

13.3. Something to close afterwards

The owner wants a record on disk of every answer the restaurant gives. A file has to be opened, written to and closed, even if something fails halfway. Create journal.scala:

package fullybooked

import cats.effect.{IO, Resource}
import java.io.PrintWriter

class Journal(writer: PrintWriter):
  def record(line: String): IO[Unit] =
    IO.blocking:
      writer.println(line)
      writer.flush()

object Journal:
  def open(path: String): Resource[IO, Journal] =
    Resource
      .make(IO.blocking(PrintWriter(path))): writer =>
        for
          _ <- IO.blocking(writer.close())
          _ <- IO.println(s"Closed $path")
        yield ()
      .map(writer => Journal(writer))

In open, the first argument to Resource.make is a program that acquires something, here a PrintWriter on the file. The second is a function that takes the writer and returns the program that releases it. Resource.make returns a Resource, which describes something that can be opened and closed. Calling use with a function gives a program that acquires the writer, runs the function and releases the writer again, whether the function finished or failed. Nothing is opened until that program runs. Java’s try with resources gives that guarantee for one block of code. A Resource is a value, so a method such as open can return one and leave its caller to decide where to use it. A Resource also has a map that changes what it holds. The .map on the last line of open carries on the chain after the indented block and turns the writer into a Journal.

Add a test for the failing case to the end of EffectTests:

  test("a resource is released when its use fails"):
    var open = 0
    val door = Resource.make(IO.delay { open = open + 1 })(_ => IO.delay { open = open - 1 })
    val result = door.use(_ => IO.raiseError(RuntimeException("fire alarm"))).attempt
    assert(result.unsafeRunSync().isLeft)
    assertEquals(open, 0)

The resource here is only a counter, which goes up by one when it’s acquired and down by one when it’s released. The function given to use returns IO.raiseError(...), a failed IO made from an exception without throwing it. After attempt has caught the failure the counter is back at zero, so the release ran. The braces around open = open + 1 are the older way of passing a block, which is still handy when the block fits on one line.

In chapter 12 a chain of a million flatMaps overflowed the stack. Add one more test, for that chain on Cats Effect, and run them all:

  test("a million flatMaps are no trouble"):
    def countdown(from: Int): IO[Unit] =
      if from == 0 then IO.unit
      else IO.delay(()).flatMap(_ => countdown(from - 1))
    countdown(1000000).unsafeRunSync()

13.4. Configuration

The journal’s file name shouldn’t be written into the code. PureConfig is a library that reads settings from a file into a case class. Add two more lines to project.scala, a dependency to go with the others and a directory for files that aren’t Scala:

//> using dep com.github.pureconfig::pureconfig-core:0.17.11
//> using resourceDir resources

Make a directory called resources in the project and put a file called application.conf in it:

restaurant = "Fully Booked"
journal = "journal.log"
store-delay = 300 milliseconds

Then create config.scala:

package fullybooked

import cats.effect.IO
import pureconfig.{ConfigReader, ConfigSource}
import scala.concurrent.duration.FiniteDuration

case class AppConfig(
    restaurant: String,
    journal: String,
    storeDelay: FiniteDuration
) derives ConfigReader

val loadConfig: IO[AppConfig] =
  IO.blocking(ConfigSource.default.loadOrThrow[AppConfig])

The derives clause from chapter 11 has the compiler write a ConfigReader, the code that reads each field of AppConfig from the setting of the same name. A name like storeDelay is matched with store-delay in the file. The ConfigReader turns 300 milliseconds into a FiniteDuration, Scala’s type for a length of time. If a setting is missing or can’t be read as its field’s type, loadOrThrow throws an exception that lists every setting with a problem. Because the call is inside IO.blocking, the exception makes loadConfig a failed IO. I want a program with bad configuration to stop at start-up and say why.

13.5. Logging

IO.println won’t do for a service, where logging needs levels and a choice of where the output goes. The usual arrangement on the JVM is the SLF4J interface with Logback behind it. log4cats puts an IO in front of that. Add two more dependencies to project.scala. The second is a Java library, so it has single colons:

//> using dep org.typelevel::log4cats-slf4j:2.8.0
//> using dep ch.qos.logback:logback-classic:1.6.7

Logback reads its settings from a file called logback.xml. Put it in resources too:

<configuration>
  <statusListener class="ch.qos.logback.core.status.NopStatusListener"/>
  <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%-5level %logger{0}: %msg%n</pattern>
    </encoder>
  </appender>
  <root level="INFO">
    <appender-ref ref="console"/>
  </root>
</configuration>

I’ve left the time out of the pattern so that the output printed in this book is the same on every run. A real service would normally include the time. The statusListener line stops Logback describing its own start-up, which it would otherwise do at length here because the scala command puts logback.xml on the classpath twice.

13.6. Putting it together

One more file uses the configuration, the logger and the journal together. Create service.scala:

package fullybooked

import cats.effect.{IO, IOApp}
import java.time.LocalTime
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger

object Evening extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]
  val eight = LocalTime.of(20, 0)

  def book(store: BookingStore, journal: Journal, party: Party): IO[Unit] =
    for
      result <- reserveIO(store, party, eight)
      _ <- logger.info(reply(party.name, result))
      _ <- journal.record(reply(party.name, result))
    yield ()

  val evening: IO[Unit] =
    for
      config <- loadConfig
      _ <- logger.info(s"${config.restaurant} is open")
      store <- IO.delay(SlowStore(config.storeDelay.toMillis))
      _ <- Journal.open(config.journal).use: journal =>
        for
          _ <- book(store, journal, Party("Silva", 2))
          _ <- book(store, journal, Party("Zielinski", 9))
        yield ()
      _ <- logger.info("Done for the night")
    yield ()

  def run: IO[Unit] =
    evening.handleErrorWith: error =>
      logger.error(s"Stopped early: ${error.getMessage}")

Each call to logger.info returns an IO[Unit], so the message is only logged if that IO is part of the program that gets run. An IO that’s built and then dropped, on a line of its own outside the for, does nothing. That’s the mistake to look for when a log message or a write goes missing.

$ scala run . --main-class fullybooked.Evening
INFO  Evening: Fully Booked is open
INFO  Evening: Silva: table 1 at 20:00
INFO  Evening: Zielinski: sorry, we can't seat 9, our largest table takes 8
Closed journal.log
INFO  Evening: Done for the night

Print the journal with cat, or with type on Windows:

$ cat journal.log
Silva: table 1 at 20:00
Zielinski: sorry, we can't seat 9, our largest table takes 8

In the program’s output, “Closed journal.log” comes before “Done for the night”, because the journal is released as soon as use finishes. Now run it with the journal somewhere that doesn’t exist. The configuration library underneath PureConfig lets a Java system property override a setting, and --java-prop sets one:

$ scala run . --main-class fullybooked.Evening --java-prop journal=/no/such/place/journal.log
INFO  Evening: Fully Booked is open
ERROR Evening: Stopped early: /no/such/place/journal.log (No such file or directory)

Opening the file failed, so the IO failed and the steps after it never ran. The handleErrorWith in run turned that into one line in the log. There’s no “Closed” line, since nothing had been opened.

13.7. Threads

The pool of threads that IOApp starts is small, with one thread for each processor core. By default every step of every IO runs on it. That’s enough for a great deal of work, but only as long as no step holds on to a thread while it waits. A call that waits for a disk or a database, or sits in a Thread.sleep, ties up one of those few threads until it returns.

Wrapping a call like that in IO.blocking tells the runtime that the body will wait. The runtime then runs the body on a thread outside the pool, which stays at full strength. reserveIO is written with IO.blocking because the store it’s given may be a SlowStore. The journal uses it for writing its file and loadConfig for reading one. Code that doesn’t wait goes in IO.delay. For waiting on purpose there’s IO.sleep, which takes a duration and holds no thread at all while the time passes. The rule I work to is that anything which might wait on the outside world goes in IO.blocking. Getting it wrong shows up late, as a program that’s slow under load and hard to diagnose.

13.8. Reading F[_]

We’ve given IO as a type argument three times, in Logger[IO], Resource[IO, Journal] and getLogger[IO]. Look up Logger in the documentation for log4cats and you’ll find it declared as Logger[F[_]], with methods such as info(message: => String): F[Unit].

The F[_] there is a type parameter that stands for a type which itself takes a parameter, such as IO, Option or List. Put IO where the F is and F[Unit] reads IO[Unit]. Libraries are written this way so that they work with more than one effect system. Their signatures also carry context bounds like [F[_]: Sync], which asks for an F that has a given Sync. That’s a type class from Cats Effect for types that can delay code, as IO.delay does. IO has a given for it, and for Async, Concurrent and Temporal, which turn up in the same position.

Some teams write all of their own code against F[_] as well, a style called tagless final. I write IO in Fully Booked, so F[_] only appears in other people’s signatures, where you can nearly always read it as IO.

13.9. Exercises

Solutions are in the book’s downloadable code, under ch13/exercises.

  1. Add a greeting setting to application.conf and to AppConfig, with a default of “Good evening”. A second line, greeting = ${?FULLY_BOOKED_GREETING}, lets an environment variable replace it when it’s set. Try both.

  2. Write chapter 12’s retry again for Cats Effect’s IO, this time waiting between attempts. IO.sleep(pause) takes a FiniteDuration, and importing scala.concurrent.duration.* lets you write one as 10.millis.

  3. A for works over Resource too. Write twoJournals(first: String, second: String): Resource[IO, (Journal, Journal)] and use it. In which order are the two files closed?

  4. Write timed[A](program: IO[A]): IO[(A, FiniteDuration)]. IO.monotonic is an IO that reads a clock.

14. Concurrency

Two people ask for the last table at eight o’clock at the same moment. In chapter 4 it took two Java threads and two carefully chosen sleeps to show what goes wrong. With IO the same race is a five-line method.

14.1. Two requests at once

As it stands, reserve decides what to book and saves it in one method. Later in this chapter the deciding has to happen without the saving, so replace reserve in availability.scala with these two:

def decide(
    bookings: List[Booking],
    party: Party,
    time: LocalTime
): Either[BookingError, Booking] =
  if party.size > Booking.largestParty then Left(BookingError.PartyTooLarge(party.size))
  else
    bestTable(party, time, bookings)
      .toRight(BookingError.NothingFree(party.size, time))
      .map(table => Booking(party, table, time, Status.Confirmed))

def reserve(store: BookingStore, party: Party, time: LocalTime): Either[BookingError, Booking] =
  decide(store.all, party, time).map: booking =>
    store.add(booking)
    booking

The new function, decide, works only from the list of bookings it’s given. It has no side effects, so it can be called any number of times from anywhere and tested without a store.

Delete the lastTable method from main.scala, threads and all. Then create concurrency.scala:

package fullybooked

import cats.effect.{Deferred, IO, IOApp, Ref}
import cats.effect.std.{Queue, Semaphore}
import cats.syntax.all.*
import java.time.LocalTime
import scala.concurrent.duration.*

val eight = LocalTime.of(20, 0)
val ibrahim = Party("Ibrahim", 7)
val castellanos = Party("Castellanos", 8)

val regulars: List[Booking] =
  Table.all
    .filter(_.number != 6)
    .map(table => Booking.walkIn(Party("Regular", table.seats), table, eight))

def lastTableRace(store: BookingStore): IO[(String, String)] =
  val first = reserveIO(store, ibrahim, eight).map(result => reply(ibrahim.name, result))
  val second = IO.sleep(100.millis) >>
    reserveIO(store, castellanos, eight).map(result => reply(castellanos.name, result))
  (first, second).parTupled

object LastTable extends IOApp.Simple:
  def slowStore: IO[BookingStore] =
    IO.blocking:
      val store = SlowStore(300)
      regulars.foreach(store.add)
      store

  def show(title: String, answers: (String, String)): IO[Unit] =
    IO.println(s"$title\n  ${answers(0)}\n  ${answers(1)}")

  def run: IO[Unit] =
    for
      raced <- slowStore.flatMap(lastTableRace)
      _ <- show("Two requests at once:", raced)
    yield ()

Several of those imports are for code still to come. As in chapter 4 the restaurant is nearly full: regulars puts a party at every table but number 6. In LastTable, slowStore makes a slow store with those bookings already in it.

In lastTableRace the Ibrahims’ reservation is first and the Castellanos party’s is second, held back a tenth of a second by IO.sleep. We can write that length of time as 100.millis because of the import of scala.concurrent.duration.*. The >> after the sleep means “then”: it runs the program on its left and after it the one on its right. On the last line, parTupled turns the tuple of two programs into a single program that runs both at the same time and gives back a tuple of their results. show prints that pair under a title. Its answers(0) is the first element of the tuple, another way of writing answers._1.

$ scala run . --main-class fullybooked.LastTable
Two requests at once:
  Ibrahim: table 6 at 20:00
  Castellanos: table 6 at 20:00

Both parties have table 6, the double booking from chapter 4. The Castellanos request read the store a tenth of a second in, while the slow store was still saving the Ibrahims’ booking.

14.2. Fibers

Each of the two programs handed to parTupled ran on a fiber. A fiber is Cats Effect’s version of a thread: a sequence of steps that runs independently of the others. It’s an object in memory and far lighter than a thread. A program can have hundreds of thousands of fibers, taking turns on the pool of real threads that chapter 13 described. A fiber that’s waiting, in IO.sleep for instance, isn’t using a thread at all.

To start and stop a fiber by hand, add this to the end of concurrency.scala and run Waiting:

object Waiting extends IOApp.Simple:
  def run: IO[Unit] =
    for
      freed <- Deferred[IO, Table]
      silva <- freed.get.map(table => s"Silva: table ${table.number} is yours").start
      moreau <- IO.sleep(1.hour).onCancel(IO.println("Moreau: gave up waiting")).start
      _ <- IO.println("Two parties are waiting")
      _ <- IO.sleep(100.millis)
      _ <- moreau.cancel
      _ <- freed.complete(Table(2, 2))
      message <- silva.joinWithNever
      _ <- IO.println(message)
    yield ()
$ scala run . --main-class fullybooked.Waiting
Two parties are waiting
Moreau: gave up waiting
Silva: table 2 is yours

Each party waits on a fiber of its own: start begins a program on a new fiber and returns at once with a handle to it. The Silvas’ fiber is waiting on freed, a Deferred made on the first line: a box that starts empty and can be filled exactly once. Its get doesn’t return until the box is full. The Moreaus’ fiber would sleep for an hour, but cancel stops it after a tenth of a second and the onCancel we attached runs on the way out. Then complete fills the box with a table, which lets the Silvas’ get return. Their message reaches us through joinWithNever, which waits for a fiber and hands back what it produced. Had that fiber been cancelled, joinWithNever would wait for ever.

The tenth of a second before the cancel is there to let both fibers get going. A fiber that’s cancelled before it has run a single step never reaches its onCancel, and on a machine with one core that happened in about half my runs without the pause.

Java has no good counterpart to cancel: interrupting a thread is a request that the code on it may or may not notice. A fiber can be cancelled between any two steps of its IO, or while it’s waiting in a step such as IO.sleep. When it is, every Resource it has open is released and every onCancel it has reached runs.

I rarely call start myself. parTupled and its relatives start the fibers, wait for them and tidy up: if one side fails, the other is cancelled and the failure is passed on. A hand-started fiber that nobody joins or cancels just keeps running, which is an easy leak to write.

14.3. State that fibers share

Part of chapter 4’s bug was a var updated by two threads at once. For a value that several fibers change, Cats Effect has Ref. Create concurrency.test.scala:

package fullybooked

import cats.effect.IO
import cats.effect.unsafe.implicits.global
import cats.syntax.all.*

class ConcurrencyTests extends munit.FunSuite:
  test("a Ref counts correctly from many fibers"):
    val program =
      for
        counter <- IO.ref(0)
        _ <- List.fill(1000)(counter.update(_ + 1)).parSequence_
        total <- counter.get
      yield total
    assertEquals(program.unsafeRunSync(), 1000)

The test makes a Ref holding 0 with IO.ref(0), and counter.update(_ + 1) is a program that adds one to it. List.fill(1000) makes a list of a thousand copies of that program for parSequence_ to run all at once, each on a fiber of its own. In Cats and Cats Effect, methods whose names end in an underscore, such as parSequence_ here and traverse_ further on, run the effects and throw away the results. Run the tests as often as you like and the answer is a thousand. Underneath, a Ref is Java’s AtomicReference with methods that return IO. When two updates collide, one of them is tried again with the new value. That means the function passed to update can run more than once, so it has to be free of side effects.

In chapter 4, checking for a table and saving the booking were two separate steps, with room for another thread in between. A Ref can make them one step with modify. Add these to the end of concurrency.scala:

class Bookings(ref: Ref[IO, List[Booking]]):
  def all: IO[List[Booking]] = ref.get

  def reserve(party: Party, time: LocalTime): IO[Either[BookingError, Booking]] =
    ref.modify: bookings =>
      decide(bookings, party, time) match
        case Right(booking) => (bookings :+ booking, Right(booking))
        case Left(error)    => (bookings, Left(error))

object Bookings:
  def starting(bookings: List[Booking]): IO[Bookings] =
    Ref.of[IO, List[Booking]](bookings).map(ref => Bookings(ref))

def lastTableFixed(bookings: Bookings): IO[(String, String)] =
  val first = bookings.reserve(ibrahim, eight).map(result => reply(ibrahim.name, result))
  val second = IO.sleep(100.millis) >>
    bookings.reserve(castellanos, eight).map(result => reply(castellanos.name, result))
  (first, second).parTupled

A Bookings keeps its list of bookings in a Ref. Bookings.starting makes that Ref with Ref.of, which is IO.ref with the types written out. The reserve method of Bookings uses modify, which is like update except that it also gives the caller an answer. The function passed to modify receives the current list and returns a pair: the new list to store, and a result for the caller. In reserve that function runs decide, then either adds the booking to the list and answers Right, or leaves the list as it was and answers with the error. The check and the save are now a single step that no other fiber can get between, which is only possible because decide needs nothing except the list.

14.4. A lock, when the state lives elsewhere

The slow store stands for a database, and the bookings in a database can’t be moved into a Ref. The traditional way to protect state like that is a lock, so that only one reservation is in progress at a time. Add this to concurrency.scala as well:

def lastTableLocked(store: BookingStore): IO[(String, String)] =
  Semaphore[IO](1).flatMap: lock =>
    val first = lock.permit.use: _ =>
      reserveIO(store, ibrahim, eight).map(result => reply(ibrahim.name, result))
    val second = IO.sleep(100.millis) >> lock.permit.use: _ =>
      reserveIO(store, castellanos, eight).map(result => reply(castellanos.name, result))
    (first, second).parTupled

A Semaphore hands out a fixed number of permits, here one. lock.permit is a Resource, so use takes the permit, runs the reservation and gives the permit back whatever happens. The _ => is there because the reservation has no use for the value a permit hands over. While the Ibrahims’ request holds the permit, the Castellanos request waits, and by the time its turn comes the Ibrahims’ booking is in the store.

Replace the LastTable object with one that runs all three versions:

object LastTable extends IOApp.Simple:
  def slowStore: IO[BookingStore] =
    IO.blocking:
      val store = SlowStore(300)
      regulars.foreach(store.add)
      store

  def show(title: String, answers: (String, String)): IO[Unit] =
    IO.println(s"$title\n  ${answers(0)}\n  ${answers(1)}")

  def run: IO[Unit] =
    for
      raced <- slowStore.flatMap(lastTableRace)
      _ <- show("Two requests at once:", raced)
      bookings <- Bookings.starting(regulars)
      fixed <- lastTableFixed(bookings)
      _ <- show("With the bookings in a Ref:", fixed)
      locked <- slowStore.flatMap(lastTableLocked)
      _ <- show("With a lock around the slow store:", locked)
    yield ()
$ scala run . --main-class fullybooked.LastTable
Two requests at once:
  Ibrahim: table 6 at 20:00
  Castellanos: table 6 at 20:00
With the bookings in a Ref:
  Ibrahim: table 6 at 20:00
  Castellanos: sorry, no table for 8 is free at 20:00
With a lock around the slow store:
  Ibrahim: table 6 at 20:00
  Castellanos: sorry, no table for 8 is free at 20:00

Both fixes give the table to the Ibrahims and tell the Castellanos party that nothing is free. The lock does it by making every reservation queue behind the one before, even when the two have nothing to do with each other. Neither fix works once there’s more than one copy of the program running, since the lock and the Ref both live in one program’s memory. Chapter 20 has the database do the job.

Add a stronger test, with twenty parties at once, to the end of ConcurrencyTests:

  test("a Ref never gives the last table to two parties"):
    val program =
      for
        bookings <- Bookings.starting(regulars)
        parties = List.tabulate(20)(n => Party(s"Party $n", 7))
        results <- parties.parTraverse(party => bookings.reserve(party, eight))
        stored <- bookings.all
      yield (results.count(_.isRight), stored.count(_.table.number == 6))
    assertEquals(program.unsafeRunSync(), (1, 1))

A line like parties = ... inside a for gives a name to an ordinary value, with = in place of the arrow. Here the value is a list of twenty parties, built by List.tabulate(20) calling its function with the numbers 0 to 19. parTraverse is traverse from chapter 11 with every element handled on its own fiber. Twenty parties of seven go for table 6 together and exactly one gets it.

14.5. A queue between fibers

When a booking is confirmed the restaurant sends a message. The code that takes the booking shouldn’t have to wait while the message goes. It can put the message on a Queue and leave another fiber to send it. Add one more object to concurrency.scala:

object Confirmations extends IOApp.Simple:
  def sender(queue: Queue[IO, String]): IO[Unit] =
    queue.take.flatMap(message => IO.println(s"Sent: $message")).replicateA_(3)

  def run: IO[Unit] =
    for
      queue <- Queue.bounded[IO, String](10)
      worker <- sender(queue).start
      _ <- List("Okonkwo", "Lindqvist", "Moreau").traverse_(name => queue.offer(s"$name, see you tonight"))
      _ <- worker.join
      _ <- IO.println("All confirmations sent")
    yield ()
$ scala run . --main-class fullybooked.Confirmations
Sent: Okonkwo, see you tonight
Sent: Lindqvist, see you tonight
Sent: Moreau, see you tonight
All confirmations sent

The queue comes from Queue.bounded and holds at most ten messages. sender runs on its own fiber, taking messages off the queue one at a time and waiting in take whenever the queue is empty. This sender stops after three messages so that the program ends: replicateA_(3) repeats a program three times. worker.join waits for the sender’s fiber to finish and returns a value saying how it ended, which the program ignores. On the other side of the queue, offer waits when the queue is full. With a bound of ten, code that fills the queue faster than the sender empties it is slowed to the sender’s speed.

14.6. Exercises

Solutions are in the book’s downloadable code, under ch14/exercises.

  1. Write lookUpAll(names: List[String]): IO[List[String]], in which each lookup sleeps for 200 milliseconds and all of them run at once. Test that three take well under 600. IO.monotonic reads a clock, and the difference between two readings is a FiniteDuration.

  2. IO.race(a, b) runs two programs, returns whichever finishes first as a Left or a Right, and cancels the other. Use it to write firstToAnswer(a: IO[String], b: IO[String]): IO[String]. merge turns an Either[String, String] into a String.

  3. Write cancel(ref: Ref[IO, List[Booking]], name: String): IO[Boolean], which removes a party’s booking in one atomic step and says whether there was one to remove.

  4. A restaurant with two greeters can show two parties to their tables at the same time. Start ten parties together, use a Semaphore with two permits to hold them to that, and report the largest number that were being seated at any moment. A pair of Refs will do for the counting, and updateAndGet on a Ref returns the new value.

  5. Write openKitchen(waiters: Int): IO[List[String]], in which every waiter’s fiber waits on one Deferred[IO, Unit] and all of them carry on when it’s completed.

15. Failure, retries and timeouts

Sending a text message means calling somebody else’s service over the network. That service can be down for a few seconds, it can be slow, and it can refuse a particular number. The confirmation sender from chapter 14 only printed its messages. We’ll rebuild it around a stand-in gateway that fails for a while or never answers.

15.1. A gateway that misbehaves

Delete the Confirmations object from concurrency.scala. Nothing in that file uses Queue any more, so change its fourth line to import cats.effect.std.Semaphore. A better Confirmations goes in a new file, confirmations.scala, which starts like this:

package fullybooked

import cats.effect.{IO, IOApp, Ref}
import cats.effect.std.Queue
import cats.syntax.all.*
import java.io.IOException
import java.util.concurrent.TimeoutException
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import scala.concurrent.duration.*

trait Messenger:
  def send(phone: Phone, text: String): IO[Unit]

class FlakyGateway(calls: Ref[IO, Map[Phone, Int]], failsTwice: Set[Phone], hangs: Set[Phone])
    extends Messenger:
  def send(phone: Phone, text: String): IO[Unit] =
    calls.updateAndGet(seen => seen.updated(phone, seen.getOrElse(phone, 0) + 1)).flatMap: seen =>
      if hangs.contains(phone) then IO.never
      else if failsTwice.contains(phone) && seen(phone) <= 2 then
        IO.raiseError(IOException("gateway unavailable"))
      else IO.unit

Some of those imports aren’t used until the file is finished. Messenger is the interface to whoever sends our texts. A real implementation would make an HTTP call to a provider. Ours is FlakyGateway, a stand-in that fails on demand so that we can see each kind of trouble without waiting for it to happen.

The gateway counts the calls made for each number in a Ref that holds a Map. On each call it uses updated to make a copy of the map with that number’s count raised by one. It stores the copy with updateAndGet, which is update that also hands back the new value. Numbers in failsTwice get an IOException on their first two calls and succeed on the third. Numbers in hangs get IO.never, a program that never finishes, which is what a call to a dead server looks like from our side.

15.2. Trying again

A gateway that’s unavailable now may well be back in a moment, so the first defence is to try again. Add this to the end of confirmations.scala:

def retrying[A](program: IO[A], attempts: Int, pause: FiniteDuration): IO[A] =
  program.handleErrorWith: error =>
    if attempts > 1 then IO.sleep(pause) >> retrying(program, attempts - 1, pause * 2)
    else IO.raiseError(error)

If program fails and there are attempts left, retrying waits and then calls itself to run the same program again, with one attempt fewer. With no attempts left, retrying fails with the last error. The pause is doubled each time. A FiniteDuration can be multiplied, so that’s pause * 2. The doubling is called backoff: a service that’s struggling isn’t helped by every client asking again at once, and a growing pause gives it a chance to recover.

Create confirmations.test.scala:

package fullybooked

import cats.effect.IO
import cats.effect.unsafe.implicits.global
import java.io.IOException
import java.util.concurrent.TimeoutException
import scala.concurrent.duration.*

class ConfirmationTests extends munit.FunSuite:
  def failing(times: Int): IO[IO[Int]] =
    IO.ref(0).map: calls =>
      calls.updateAndGet(_ + 1).flatMap: n =>
        if n <= times then IO.raiseError(IOException(s"attempt $n failed")) else IO.pure(n)

  test("retrying carries on until the program succeeds"):
    val result = failing(2).flatMap(program => retrying(program, attempts = 3, pause = 1.milli))
    assertEquals(result.unsafeRunSync(), 3)

  test("retrying gives up with the last error"):
    val result = failing(5).flatMap(program => retrying(program, attempts = 3, pause = 1.milli))
    assertEquals(result.attempt.unsafeRunSync().left.map(_.getMessage), Left("attempt 3 failed"))

The helper failing(2) makes a program that fails on its first two runs and then gives the number of the run. The return type of failing is an IO of an IO. Running the outer one creates the counter and gives back the inner one, the program that uses it. Run the tests.

Retrying only helps with failures that might go away. A gateway that’s down may come back, but a number it has refused will be refused again. A retry can also do the job twice: if the gateway sent the text and its reply was lost on the way back, our second attempt sends another. A second confirmation text does no harm, but where a repeat would matter the service being called has to be able to recognise a duplicate. That usually means we send the same identifier with every attempt.

15.3. Giving up

A server that never answers is worse than one that says no, because the program waiting on it is never told anything. Add three tests to the end of ConfirmationTests and run the tests:

  test("a program that takes too long is cancelled"):
    var cancelled = false
    val slow = IO.sleep(1.hour).onCancel(IO.delay { cancelled = true })
    val result = slow.timeout(50.millis).attempt.unsafeRunSync()
    assert(result.left.exists(error => error.isInstanceOf[TimeoutException]))
    assert(cancelled)

  test("timeoutTo answers with something else"):
    val slow = IO.sleep(1.hour).map(_ => "answer")
    assertEquals(slow.timeoutTo(50.millis, IO.pure("gave up")).unsafeRunSync(), "gave up")

  test("a timeout can't cut IO.blocking short, but it can IO.interruptible"):
    val stuck = IO.blocking { Thread.sleep(300); "late" }
    assertEquals(stuck.timeoutTo(50.millis, IO.pure("gave up")).unsafeRunSync(), "late")
    val interrupted = IO.interruptible { Thread.sleep(300); "late" }
    assertEquals(interrupted.timeoutTo(50.millis, IO.pure("gave up")).unsafeRunSync(), "gave up")

Calling timeout on a program puts a limit on it. When the time runs out the slow program is cancelled and the result is a failed IO holding a TimeoutException. The first of these tests checks both, with an onCancel to see the cancellation. Its assertion on the result uses isInstanceOf, Scala’s instanceof, inside left.exists, which asks whether there’s a Left whose value satisfies a condition. In the second test, timeoutTo runs another program where timeout would fail.

I put a timeout on every call that leaves the process. Client libraries often have a default of several minutes or none at all. With no limit, a service can end up with every request waiting on a server that’s never going to answer.

A timeout works by cancelling the program it’s limiting, which chapter 14 said can happen between steps or while a fiber waits in something like IO.sleep. A call inside IO.blocking is a single step with a real thread stuck in it, so there’s no point at which to cancel. In the third test timeoutTo has to wait for the sleep to finish and gives the late answer. IO.interruptible is the version of IO.blocking that interrupts the thread when the fiber is cancelled. That works for code which responds to being interrupted, as Thread.sleep does. Some blocking I/O doesn’t, so with a blocking Java client I set the client’s own timeouts as well.

15.4. Cleaning up whatever happens

A program can finish, fail or be cancelled. Code that must run in all three cases goes in guarantee. Add one more test to ConfirmationTests and run the tests:

  test("guarantee runs after success, failure and cancellation"):
    val program =
      for
        ran <- IO.ref(0)
        _ <- IO.unit.guarantee(ran.update(_ + 1))
        _ <- IO.raiseError(IOException("no")).guarantee(ran.update(_ + 1)).attempt
        _ <- IO.never.guarantee(ran.update(_ + 1)).timeoutTo(50.millis, IO.unit)
        count <- ran.get
      yield count
    assertEquals(program.unsafeRunSync(), 3)

The counter reaches three: once for IO.unit, once for the IO.raiseError and once for the IO.never that the timeout cancels. guarantee does the job of Java’s finally, and for code that should run in only one of the cases there are onError and the onCancel we’ve used already.

15.5. Sending the confirmations

Add the rest of confirmations.scala, the sender itself:

def confirm(messenger: Messenger, logger: Logger[IO], booking: Booking): IO[Unit] =
  booking.party.phone match
    case None =>
      logger.info(s"${booking.party.name}: no number, nothing sent")
    case Some(phone) =>
      val text = s"${booking.party.name}, table ${booking.table.number} at ${booking.time}"
      val sending = messenger.send(phone, text).timeout(200.millis)
      retrying(sending, attempts = 3, pause = 20.millis)
        .flatMap(_ => logger.info(s"${booking.party.name}: sent"))
        .recoverWith:
          case _: TimeoutException => logger.warn(s"${booking.party.name}: gateway didn't answer")
          case error: IOException  => logger.warn(s"${booking.party.name}: ${error.getMessage}")

def worker(
    queue: Queue[IO, Option[Booking]],
    messenger: Messenger,
    logger: Logger[IO],
    handled: Ref[IO, Int]
): IO[Unit] =
  queue.take.flatMap:
    case None => IO.unit
    case Some(booking) =>
      confirm(messenger, logger, booking).guarantee(handled.update(_ + 1)) >>
        worker(queue, messenger, logger, handled)

object Confirmations extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def numbersOf(name: String): Set[Phone] =
    tonight.filter(_.party.name == name).flatMap(_.party.phone).toSet

  def run: IO[Unit] =
    val confirmed = tonight.filter(_.status == Status.Confirmed)
    for
      calls <- IO.ref(Map[Phone, Int]())
      handled <- IO.ref(0)
      gateway = FlakyGateway(calls, numbersOf("Lindqvist"), numbersOf("Moreau"))
      queue <- Queue.bounded[IO, Option[Booking]](10)
      sender <- worker(queue, gateway, logger, handled).start
      _ <- confirmed.traverse_(booking => queue.offer(Some(booking)))
      _ <- queue.offer(None)
      _ <- sender.joinWithNever
      count <- handled.get
      attempts <- calls.get
      _ <- logger.info(s"Handled $count bookings with ${attempts.values.sum} calls to the gateway")
    yield ()

In confirm, the timeout is inside the retry, so each of up to three attempts gets a fifth of a second. Putting the timeout outside would give one limit for the whole job, which is exercise 4. The recoverWith at the end takes a case lambda and deals only with the failures that match one of its cases. Its first pattern, case _: TimeoutException, matches by type without giving the exception a name. When every attempt has timed out or met an IOException, confirm logs a warning and finishes normally, because one undeliverable message mustn’t stop the rest. An exception of any other type is still a failure.

The queue carries Option[Booking] so that run can tell the worker when to stop. A Some holds a booking to confirm, and on None the worker stops calling itself and ends. Each confirmation goes through guarantee, so the handled counter goes up even for a booking whose confirm fails. The run method calls start to put the worker on a fiber of its own, then puts tonight’s confirmed bookings and the final None on the queue. It waits in joinWithNever for the worker to finish. Had the worker failed with an exception we didn’t plan for, joinWithNever would fail the whole program with it. Run Confirmations:

$ scala run . --main-class fullybooked.Confirmations
INFO  Confirmations: Okonkwo: sent
INFO  Confirmations: Lindqvist: sent
WARN  Confirmations: Moreau: gateway didn't answer
INFO  Confirmations: Fitzgerald: no number, nothing sent
INFO  Confirmations: Handled 4 bookings with 7 calls to the gateway

The Okonkwo message went first time. The Lindqvists’ message failed twice and went on the third call, which shows only in the total. The Moreaus’ number hangs, so all three attempts timed out and the log has a warning. With no number for the Fitzgeralds, that’s one call, three and three: the seven in the last line.

15.6. Two kinds of failure

Every failure in this chapter has been an exception inside a failed IO, where chapter 9 argued for errors as values. A Left is still for what chapter 9 called failures that are part of the job, such as a full restaurant. The Left is in the return type, so the compiler makes the caller look at it. There’d be no point retrying a call that returned one. A failed IO is for trouble with the machinery, such as a network that’s down. It travels up through every flatMap without being mentioned, until code like confirm retries it or logs it.

With attempt, a failed IO becomes a successful one holding a Left, and IO.fromEither goes the other way when the Left holds an exception. Where several steps each return an IO of an Either, chaining them gets long-winded. Many codebases use a wrapper from Cats called EitherT for that. I match on the Either by hand, as worker matches on its Option, until a method has enough steps to make it worth the extra type.

15.7. Exercises

Solutions are in the book’s downloadable code, under ch15/exercises.

  1. Define case class Refused(reason: String) extends Exception(reason) and write retryingTransient, which behaves like retrying except that a Refused is never retried. The (reason) after Exception passes the reason to its constructor. handleErrorWith accepts a case lambda too, but its cases have to cover every error.

  2. Write retryingCapped[A](program: IO[A], attempts: Int, pause: FiniteDuration, most: FiniteDuration): IO[A], in which the pause doubles as before but never grows beyond most. A FiniteDuration has a min method.

  3. Write sendWithFallback(first: Messenger, second: Messenger, phone: Phone, text: String): IO[String], which tries a second messenger when the first fails and returns "first" or "second" to say which one worked. In a test, new Messenger: followed by an indented send makes a messenger on the spot.

  4. Write withinLimit[A](program: IO[A], limit: FiniteDuration): IO[A], which retries up to a hundred times, starting with a pause of 5 milliseconds, but gives the whole job a single time limit.

16. Streams with fs2

An agency takes some of the restaurant’s bookings over the phone. Each night it sends them across as a file with one booking to a line: name, party size, time and phone number. Create nightly.csv in the project directory, beside the Scala files:

Hughes,4,18:30,07700 900201
Banerjee,2,19:00,
Olawale,six,19:15,07700 900203
Kowalczyk,3,19:30,07700 900204
,2,19:45,07700 900205
Mbeki,8,20:00,07700 900206
Tanaka,8,20:00,07700 900207
Dubois,2,7pm,call the hotel

Three of the eight lines have something wrong with them. Lines 6 and 7 are two parties who both want a table for eight at eight o’clock. The import has to book what it can and say what was wrong with the rest. It also has to cope with a file of any length, so reading every line into a List before starting is out.

fs2 is the streaming library that goes with Cats Effect. Two of Part IV’s libraries are built on fs2: http4s carries the body of a request or a response as an fs2 stream, and doobie can return the rows of a query as one.

16.1. Reading the file

Add the dependency to project.scala, under the one for Cats Effect:

//> using dep co.fs2::fs2-io:3.14.0

Then create importer.scala. This first version only prints the file:

package fullybooked

import cats.effect.{ExitCode, IO, IOApp}
import fs2.Stream
import fs2.io.file.{Files, Path}
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger

object ImportBookings extends IOApp.Simple:
  def run: IO[Unit] =
    Files[IO]
      .readUtf8Lines(Path("nightly.csv"))
      .zipWithIndex
      .filter((line, _) => line.trim.nonEmpty)
      .evalMap((line, index) => IO.println(s"${index + 1}: $line"))
      .compile
      .drain
$ scala run . --main-class fullybooked.ImportBookings
1: Hughes,4,18:30,07700 900201
2: Banerjee,2,19:00,
3: Olawale,six,19:15,07700 900203
4: Kowalczyk,3,19:30,07700 900204
5: ,2,19:45,07700 900205
6: Mbeki,8,20:00,07700 900206
7: Tanaka,8,20:00,07700 900207
8: Dubois,2,7pm,call the hotel

The imports are for the finished file. Read the chain in run from the top, as you would a chain of calls on a List: read the lines, number them with zipWithIndex, drop the blank ones, print what’s left. The numbers are Longs counting from 0. I filter after numbering so that the numbers still match the file when a blank line turns up in the middle. There’s usually one blank at the end as well. If the file finishes with a line break, as most editors leave it, readUtf8Lines reports the empty text after it as a last line.

The type that readUtf8Lines returns is Stream[IO, String], a stream of strings that needs IO to produce them. Where map takes an ordinary function, evalMap takes one that returns an IO and runs that IO for each element in turn. At the end, compile.drain turns the stream into an IO[Unit]. Running that IO pulls every element through the stream, so that the effects happen, and throws the elements away.

Nothing in the file is read until IOApp runs that IO. Even then the file is never in memory as a whole. fs2 reads a piece of it, splits the piece into lines, passes them down to the println and lets them go before it reads the next piece.

16.2. A stream is a description

Create streams.test.scala, with imports for all the tests in this chapter, and run the tests:

package fullybooked

import cats.effect.IO
import cats.effect.unsafe.implicits.global
import fs2.Stream
import fs2.io.file.{Files, Path}
import scala.concurrent.duration.*

class StreamTests extends munit.FunSuite:
  test("a stream without effects can be endless"):
    val sevens = Stream.iterate(1)(_ + 1).filter(_ % 7 == 0).take(3)
    assertEquals(sevens.toList, List(7, 14, 21))

  test("a stream of effects does nothing until it's compiled and run"):
    val program =
      for
        counter <- IO.ref(0)
        ticks = Stream.eval(counter.update(_ + 1)).repeat.take(3)
        before <- counter.get
        _ <- ticks.compile.drain
        after <- counter.get
      yield (before, after)
    assertEquals(program.unsafeRunSync(), (0, 3))

The first test takes three multiples of seven from a stream of every whole number from 1 upwards. Stream.iterate(1)(_ + 1) has no end. That’s allowed because a stream produces an element only when something downstream asks for one, and take(3) stops asking after the third. A stream with no effects in it can be turned straight into a list with toList.

In the second test ticks is another endless stream, one that adds 1 to a counter for every element, cut off after three. Stream.eval makes a one-element stream out of an IO and repeat goes round it again and again. The counter still read 0 after ticks had been built, because a stream of effects is a description, as an IO is. It reached 3 only when the IO from compile.drain was run.

A stream of effects goes through compile before it can be run. The method called on compile gives an IO and decides what that IO returns: drain returns nothing, toList returns every element, count returns how many there were, last returns the final one in an Option, and fold combines them as foldLeft does.

16.3. The import

Add these to importer.scala, between the imports and ImportBookings:

def requestFrom(line: String): BookingRequest =
  val fields = line.split(",", -1).toList.map(_.trim).padTo(4, "")
  BookingRequest(fields(0), fields(1), fields(2), fields(3))

case class Rejected(lineNumber: Long, errors: List[BookingError])

def importLine(bookings: Bookings, line: String, lineNumber: Long): IO[Either[Rejected, Booking]] =
  validate(requestFrom(line)) match
    case Left(errors) =>
      IO.pure(Left(Rejected(lineNumber, errors.toList)))
    case Right((party, time)) =>
      bookings.reserve(party, time).map(_.left.map(error => Rejected(lineNumber, List(error))))

def importFile(bookings: Bookings, path: Path): Stream[IO, Either[Rejected, Booking]] =
  Files[IO]
    .readUtf8Lines(path)
    .zipWithIndex
    .filter((line, _) => line.trim.nonEmpty)
    .evalMap((line, index) => importLine(bookings, line, index + 1))

The first method, requestFrom, turns a line into the four strings of chapter 9’s BookingRequest. Java’s split drops empty fields from the end of a line unless it’s given -1. padTo(4, "") fills a short line out with empty strings, so fields(3) is always there. A real CSV file can have commas inside quoted fields. For those I’d use a CSV library such as fs2-data-csv.

In importLine the work is done by code from earlier chapters. Chapter 9’s validate collects every problem with the four strings. A request that passes goes to chapter 14’s Bookings, where the reservation is made in one atomic step. A refusal from either of them ends up in a Rejected with the number of its line.

In importFile the stream is the one the first version had, with importLine where the println was. The method returns it without compiling it, so the caller decides what happens to the results. Replace ImportBookings with a version that prints each result and counts the bookings:

object ImportBookings extends IOApp:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def report(result: Either[Rejected, Booking]): IO[Unit] =
    result match
      case Right(booking) =>
        IO.println(label(booking))
      case Left(Rejected(lineNumber, errors)) =>
        IO.println(s"Line $lineNumber: ${errors.map(explain).mkString("; ")}")

  def run(args: List[String]): IO[ExitCode] =
    val path = Path(args.headOption.getOrElse("nightly.csv"))
    val importing =
      for
        bookings <- Bookings.starting(List())
        imported <- importFile(bookings, path).evalTap(report).filter(_.isRight).compile.count
        _ <- logger.info(s"Imported $imported bookings from $path")
      yield ExitCode.Success
    importing.handleErrorWith: error =>
      logger.error(s"Couldn't import $path: $error").map(_ => ExitCode.Error)

Each result is printed by evalTap(report), which runs an effect for each element and passes the element on unchanged. The filter and compile.count after the evalTap leave a single number, the bookings made.

This object extends IOApp where the others have extended IOApp.Simple. Its run receives the command-line arguments and returns an ExitCode for the process to exit with. With no argument it reads nightly.csv:

$ scala run . --main-class fullybooked.ImportBookings
18:30 table 3: Hughes, party of 4
19:00 table 1: Banerjee, party of 2
Line 3: 'six' isn't a number of people
19:30 table 4: Kowalczyk, party of 3
Line 5: we need a name for the booking
20:00 table 6: Mbeki, party of 8 (deposit taken)
Line 7: no table for 8 is free at 20:00
Line 8: '7pm' isn't a time, try 19:30; 'call the hotel' isn't a phone number
INFO  ImportBookings: Imported 4 bookings from nightly.csv

Three lines failed validation, line 8 on two counts. The Mbekis got table 6 and the Tanakas, one line later, were refused it. The order of the file decides that, because evalMap finishes one line before it starts the next.

Add a test for the import to the end of StreamTests:

  test("the import books what it can and reports the rest"):
    val lines = List("Hughes,4,18:30,", "", "Olawale,six,19:15,", "Mbeki,8,20:00,", "Tanaka,8,20:00,")
    val program = Files[IO].tempFile.use: path =>
      for
        _ <- Stream.emits(lines).through(Files[IO].writeUtf8Lines(path)).compile.drain
        bookings <- Bookings.starting(List())
        results <- importFile(bookings, path).compile.toList
      yield results.map(_.left.map(_.lineNumber).map(_.party.name))
    assertEquals(program.unsafeRunSync(), List(Right("Hughes"), Left(3L), Right("Mbeki"), Left(5L)))

The test writes five lines to a temporary file, imports it and checks which lines were booked. The temporary file is a Resource, so it’s deleted when use finishes. A stream does the writing too: Stream.emits(lines) makes a stream from the list and through sends it into a pipe. A pipe is what fs2 calls a function from one stream to another. The pipe from writeUtf8Lines writes each element as a line and emits nothing itself, hence the drain. Because the second line of the file is blank, the rejections are on lines 3 and 5, written 3L and 5L as Longs.

16.4. A file bigger than memory

Create bigfile.scala, which writes a CSV file too big to hold in a small heap and then reads it back in two ways:

package fullybooked

import cats.effect.{IO, IOApp}
import fs2.Stream
import fs2.io.file.{Files, Path}

object BigFile extends IOApp.Simple:
  def line(n: Int): String =
    if n % 10 == 0 then s"Party $n,lots,19:00," else s"Party $n,2,19:00,"

  def run: IO[Unit] =
    Stream
      .range(0, 3000000)
      .map(line)
      .chunkN(4096)
      .unchunks
      .through(Files[IO].writeUtf8Lines(Path("big.csv")))
      .compile
      .drain

@main def checkAtOnce(): Unit =
  val lines = java.nio.file.Files.readAllLines(java.nio.file.Path.of("big.csv"))
  println(s"${lines.size} lines read")

object CheckFile extends IOApp.Simple:
  def run: IO[Unit] =
    Files[IO]
      .readUtf8Lines(Path("big.csv"))
      .filter(line => line.trim.nonEmpty)
      .map(line => validate(requestFrom(line)))
      .filter(_.isRight)
      .compile
      .count
      .flatMap(valid => IO.println(s"$valid valid requests"))

Run BigFile with scala run . --main-class fullybooked.BigFile. Stream.range(0, 3000000) counts from 0 up to, but not including, three million. The run takes a few seconds and leaves a big.csv of three million lines, nearly 70 megabytes, in which every tenth line has a party size of “lots”.

The @main method checkAtOnce reads the file the way a short Java program might, all of it into a list of lines. It spells out java.nio.file.Files and java.nio.file.Path in full because the short names in this file belong to fs2’s own versions. Run it with the option --java-opt -Xmx64m to limit the JVM to 64 megabytes of heap:

$ scala run . --main-class fullybooked.checkAtOnce --java-opt -Xmx64m
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

A stack trace follows it. Now run CheckFile with the same limit. It puts every line through requestFrom and validate:

$ scala run . --main-class fullybooked.CheckFile --java-opt -Xmx64m
2700000 valid requests

No more than a few thousand lines were in memory at any moment, however long the file is. That only holds while nothing downstream collects the elements. Change count to toList in CheckFile and it runs out of memory as checkAtOnce did. Delete big.csv when you’ve finished with it.

16.5. Chunks

The two lines of BigFile that say chunkN(4096) and unchunks change how the stream’s elements are grouped. Add a test after the import test:

  test("elements travel in chunks"):
    assertEquals(Stream.range(0, 4).chunks.map(_.size).toList, List(1, 1, 1, 1))
    assertEquals(Stream.emits(List(0, 1, 2, 3)).chunks.map(_.size).toList, List(4))
    assertEquals(Stream.range(0, 10).chunkN(4).map(_.size).toList, List(4, 4, 2))

Elements move through a stream in batches called chunks, and the chunks method lets a test look at them. Stream.range produces its numbers one to a chunk, where Stream.emits puts everything it’s given into a single chunk. A file is read 64 kilobytes at a time, which comes to a few thousand of our lines. The third assertion regroups ten elements with chunkN(4) and gets chunks of 4, 4 and 2. BigFile follows its chunkN(4096) with unchunks, to get back from a stream of chunks to a stream of elements with the new grouping kept.

The grouping matters when something is done once per chunk, as writing is: writeUtf8Lines makes one write to the file for each chunk it receives. Without those two lines BigFile would make three million separate writes. I timed both versions writing 200,000 lines. The run with one line to a chunk made 200,000 writes. It took several times as long as the run with chunks of 4,096, which made about fifty.

evalMap and evalTap leave chunks of one behind them, since they run an effect for each element. A write that comes after one of them gets a chunk per element again unless a chunkN and an unchunks are put in between.

16.6. When something fails

Add two more tests to the end of StreamTests, about streams that fail:

  test("a failure ends the stream unless each element handles its own"):
    val numbers = Stream(1, 2, 0, 4)
    val stopped = numbers.evalMap(n => IO.delay(12 / n)).compile.toList.attempt
    assert(stopped.unsafeRunSync().isLeft)

    val carriedOn = numbers.evalMap(n => IO.delay(12 / n).attempt).compile.toList
    assertEquals(carriedOn.unsafeRunSync().map(_.toOption), List(Some(12), Some(6), None, Some(3)))

  test("a file that isn't there fails the stream when it runs"):
    val program =
      for
        bookings <- Bookings.starting(List())
        results <- importFile(bookings, Path("no-such-file.csv")).compile.toList.attempt
      yield results.left.map(_.getClass.getSimpleName)
    assertEquals(program.unsafeRunSync(), Left("NoSuchFileException"))

The first test makes a stream of four numbers with Stream(1, 2, 0, 4) and divides 12 by each of them inside IO.delay. The third division throws, that effect fails and the stream fails with it. The 4 is never reached, and the IO that compile.toList made fails too, losing the two good answers.

In the second half attempt turns each element’s failure into a value and the stream carries on to the 4. The import never needed attempt for a bad line, because importLine already returns a Left for one and the stream goes on to the next line. An exception thrown inside importLine would still end it, as would a failure to read the file.

The last test gives importFile a file that doesn’t exist. Nothing looks for the file until the stream runs, so importFile returns a stream as usual and the NoSuchFileException appears when that stream is run. In ImportBookings the handleErrorWith around the whole program catches it:

$ scala run . --main-class fullybooked.ImportBookings -- tomorrow.csv
ERROR ImportBookings: Couldn't import tomorrow.csv: java.nio.file.NoSuchFileException: tomorrow.csv

The exit status of that run is 1, from the ExitCode.Error in the handler. Whatever runs the import every night can check that. A stream that fails or is cancelled part-way through still closes the file it was reading, because fs2 opens the file as a Resource.

16.7. Several at once

A stream can also run several effects at once, for work where one element at a time is too slow. One more test for the end of StreamTests:

  test("parEvalMap runs several at once and keeps the order"):
    val program =
      for
        busy <- IO.ref(0)
        most <- IO.ref(0)
        doubled <- Stream
          .range(1, 9)
          .parEvalMap(3): n =>
            for
              now <- busy.updateAndGet(_ + 1)
              _ <- most.update(_.max(now))
              _ <- IO.sleep(50.millis)
              _ <- busy.update(_ - 1)
            yield n * 2
          .compile
          .toList
        atOnce <- most.get
      yield (doubled, atOnce)
    assertEquals(program.unsafeRunSync(), (List(2, 4, 6, 8, 10, 12, 14, 16), 3))

With parEvalMap(3), up to three effects run at a time. Each effect in the test adds itself to busy, a count of those in progress, and keeps the highest count seen so far in most. Then it sleeps for a twentieth of a second. The highest count was three. The results still came out in the order the elements went in. parEvalMapUnordered gives them in the order they finish, which can be quicker when one slow element would hold up the rest.

I haven’t used parEvalMap in the import. With several lines in progress at once, which of the two parties got table 6 would depend on timing and no longer on the order of the file. I’d use parEvalMap where the elements have nothing to do with one another, for example to send a confirmation for every booking the import made.

16.8. Exercises

Solutions are in the book’s downloadable code, under ch16/exercises.

  1. Write seatings, a stream of every quarter of an hour from opens to lastOrders, using Stream.iterate. A stream has takeWhile, and LocalTime has plusMinutes and isAfter.

  2. Write rejections(bookings: Bookings, path: Path): IO[Map[String, Int]], which counts how many times each reason for rejecting a line came up, in one pass over the file. collect and flatMap work on a stream, and compile.fold takes a starting value and a function as foldLeft does.

  3. Write writeRejected(bookings: Bookings, from: Path, to: Path): IO[Unit], which puts a line of text for each rejected line into a second file.

  4. Write importFileInParallel, which is importFile with parEvalMap(4), and import nightly.csv with it. The Mbekis will probably still get table 6. Why can’t you rely on that?

  5. Take chunkN and unchunks out of BigFile, reduce the range to 200,000 and time it with and without them. IO.monotonic reads a clock. There’s no solution for this one, but my figures are in the chapter.

17. Direct style

Chapter 13 gave a rule that anything which waits goes in IO.blocking. The rule exists because a waiting thread can’t be used for anything else, and real threads are heavy: each has a stack of its own and takes time to start. Fibers and IO.sleep are Cats Effect’s way round that.

Java 21 added virtual threads, which a program can have by the hundred thousand. We’ll write the last-table race and the confirmation sender again on virtual threads, without IO, and then compare the two ways of working.

17.1. A hundred thousand threads

The code in this chapter uses Ox, a library from a company called SoftwareMill. Add it to project.scala, under the line for fs2:

//> using dep com.softwaremill.ox::core:1.1.0

Then create direct.scala, with imports for the whole chapter and a first program that only uses Java’s own API:

package fullybooked

import java.io.IOException
import java.time.LocalTime
import java.util.concurrent.TimeoutException
import java.util.concurrent.atomic.AtomicInteger
import ox.{either, fork, mapPar, par, sleep, timeout, Fork, Ox}
import ox.either.ok
import ox.resilience.retry
import ox.scheduling.Schedule
import scala.concurrent.duration.*

@main def manyThreads(): Unit =
  val seated = AtomicInteger(0)
  val threads = List.fill(100000):
    Thread.ofVirtual().start: () =>
      Thread.sleep(1000)
      seated.incrementAndGet()
  threads.foreach(_.join())
  println(s"${seated.get} parties seated")
$ scala run . --main-class fullybooked.manyThreads
100000 parties seated

The program starts its threads through Java’s Thread.ofVirtual(). A Scala lambda stands in for the Runnable. Every one of the hundred thousand threads slept for a full second in plain Thread.sleep, yet the whole run takes about two seconds on my machine.

A virtual thread is an object on the heap. It runs on one of a small pool of real threads until it blocks, in a sleep, a lock or a read from a socket. At that point the JVM takes it off the real thread and puts another virtual thread on. That’s the arrangement Cats Effect has for fibers, done by the JVM for ordinary blocking code.

So a method can go back to blocking while it waits and returning its answer when it has one. Code written that way, with no wrapper type around the result, is called direct style.

17.2. The last table, a third time

The reserve in availability.scala is already in direct style: it blocks while the slow store sleeps and then returns a plain Either. Add these to the end of direct.scala:

def lastTableDirect(store: BookingStore): (String, String) =
  def first = reply(ibrahim.name, reserve(store, ibrahim, eight))
  def second =
    sleep(100.millis)
    reply(castellanos.name, reserve(store, castellanos, eight))
  par(first, second)

class DirectBookings(private var bookings: List[Booking]):
  def all: List[Booking] = synchronized(bookings)

  def reserve(party: Party, time: LocalTime): Either[BookingError, Booking] =
    synchronized:
      decide(bookings, party, time).map: booking =>
        bookings = bookings :+ booking
        booking

def lastTableDirectFixed(bookings: DirectBookings): (String, String) =
  def first = reply(ibrahim.name, bookings.reserve(ibrahim, eight))
  def second =
    sleep(100.millis)
    reply(castellanos.name, bookings.reserve(castellanos, eight))
  par(first, second)

@main def directLastTable(): Unit =
  def show(title: String, answers: (String, String)): Unit =
    println(s"$title\n  ${answers(0)}\n  ${answers(1)}")

  val store = SlowStore(300)
  regulars.foreach(store.add)
  show("Two requests at once:", lastTableDirect(store))
  show("With a synchronized class:", lastTableDirectFixed(DirectBookings(regulars)))
$ scala run . --main-class fullybooked.directLastTable
Two requests at once:
  Ibrahim: table 6 at 20:00
  Castellanos: table 6 at 20:00
With a synchronized class:
  Ibrahim: table 6 at 20:00
  Castellanos: sorry, no table for 8 is free at 20:00

Compare lastTableDirect with chapter 14’s lastTableRace. Both have a first, a second held back by a sleep, and a last line that runs the two together. The map and >> have gone, because there’s nothing to chain: each reservation is a method call that returns when it’s finished. Ox’s par runs its two arguments on a virtual thread each and gives back both results as a tuple.

In chapter 14 first and second were vals, each holding an IO that hadn’t run. Here they have to be defs, which is easy to miss. A val would make the reservation there and then, on the current thread, before par was ever called. par takes its arguments by name, like whenOpen in chapter 7, so a def runs when par uses it.

The first pair of answers in the output is the double booking from chapters 4 and 14 again, this time on blocking threads. DirectBookings fixes it as a Java class would, with a var and synchronized around the check and the save. Putting private var in the parameter list makes bookings a field that only the class can touch. In Scala synchronized is a method that takes the block to lock around. Inside the lock, decide does the checking.

Until Java 24 a virtual thread that blocked inside synchronized kept hold of its real thread. If you’ve read advice against mixing the two, that’s what it was about.

Create direct.test.scala, with imports for the tests still to come, and run the tests:

package fullybooked

import java.io.IOException
import java.util.concurrent.atomic.AtomicInteger
import ox.{fork, mapPar, sleep, supervised}
import scala.concurrent.duration.*
import scala.util.Try

class DirectTests extends munit.FunSuite:
  test("a synchronized class never gives the last table to two parties"):
    val bookings = DirectBookings(regulars)
    val parties = List.tabulate(20)(n => Party(s"Party $n", 7))
    val results = parties.mapPar(20)(party => bookings.reserve(party, eight))
    assertEquals(results.count(_.isRight), 1)
    assertEquals(bookings.all.count(_.table.number == 6), 1)

The test sends twenty parties after table 6 with mapPar(20). That’s map with up to twenty elements in progress at once, each on a virtual thread of its own.

17.3. Scopes

Chapter 14 said that a fiber started by hand and forgotten is an easy leak to write. Ox’s fork can only start a thread inside a scope, which says where the thread’s life ends. Add a test to the end of DirectTests in which one of two threads fails:

  test("a scope ends all of its forks when one of them fails"):
    val finished = AtomicInteger(0)
    val result = Try:
      supervised:
        fork:
          sleep(1.hour)
          finished.incrementAndGet()
        fork:
          sleep(50.millis)
          throw IOException("fire alarm")
        sleep(1.hour)
    assertEquals(result.failed.map(_.getMessage), Try("fire alarm"))
    assertEquals(finished.get, 0)

In the test, supervised opens a scope and each fork starts a virtual thread inside it. A scope ends when its block ends: any fork still running is interrupted and the scope waits for it to stop. The sleep(1.hour) on the block’s last line is only there to keep this scope open. Like the one in lastTableDirect, it’s Ox’s own sleep, which takes a duration.

After a twentieth of a second the fork with the short sleep throws. The scope interrupts everything else in it, waits, and throws the same exception itself. The first fork never gets to its last line, which is why finished stays at 0. The Try around the scope catches the exception. Calling failed on a failed Try gives a successful one holding the exception, so that the test can compare the message. par and mapPar open scopes of their own. Tying a thread’s life to a block of code like this is called structured concurrency.

Ox interrupts a fork with Java’s Thread.interrupt. A thread that’s blocked in a sleep or a wait gets an InterruptedException and unwinds.

A method that forks without opening a scope of its own has to be called from inside one. If you added this to direct.scala, it wouldn’t compile:

def reserveLater(
    bookings: DirectBookings,
    party: Party,
    after: FiniteDuration
): Fork[Either[BookingError, Booking]] =
  fork:
    sleep(after)
    bookings.reserve(party, eight)
$ scala compile .
[error] ./direct.scala:61:35
[error] This operation must be run within a `supervised` or `supervisedError` block. Alternatively, you must require that the enclosing method is run within a scope, by adding a `using Ox` parameter list.
[error]     bookings.reserve(party, eight)
[error]                                   ^
Compilation failed

The working version takes the message’s second suggestion, a using Ox parameter list. It goes after directLastTable, in place of the broken one if you tried it:

def reserveLater(bookings: DirectBookings, party: Party, after: FiniteDuration)(using
    Ox
): Fork[Either[BookingError, Booking]] =
  fork:
    sleep(after)
    bookings.reserve(party, eight)

The using Ox is a using parameter of the kind chapter 11 introduced, without a name this time because nothing refers to it. supervised supplies a given Ox inside its block and fork asks for one, so the compiler can tell whether a fork has a scope to belong to. A value that stands for permission to do something is called a capability. reserveLater returns a Fork, a handle to the running thread. Calling join on the handle waits for the thread’s result. Add a test to the end of DirectTests that calls reserveLater from a scope and joins both forks:

  test("a method that forks is called inside a scope"):
    val bookings = DirectBookings(regulars)
    val answers = supervised:
      val late = reserveLater(bookings, castellanos, 50.millis)
      val early = reserveLater(bookings, ibrahim, 0.millis)
      (early.join(), late.join())
    assert(answers(0).isRight)
    assert(answers(1).isLeft)

17.4. Timeouts and retries

Add a direct-style version of chapter 15’s confirmation sender to the end of direct.scala:

trait DirectMessenger:
  def send(phone: Phone, text: String): Unit

class FlakyDirectGateway(failsTwice: Set[Phone], hangs: Set[Phone]) extends DirectMessenger:
  private var calls = Map[Phone, Int]()

  def totalCalls: Int = synchronized(calls.values.sum)

  def send(phone: Phone, text: String): Unit =
    val seen = synchronized:
      calls = calls.updated(phone, calls.getOrElse(phone, 0) + 1)
      calls(phone)
    if hangs.contains(phone) then sleep(1.hour)
    else if failsTwice.contains(phone) && seen <= 2 then throw IOException("gateway unavailable")

def confirmDirect(messenger: DirectMessenger, booking: Booking): String =
  val name = booking.party.name
  booking.party.phone match
    case None => s"$name: no number, nothing sent"
    case Some(phone) =>
      val text = s"$name, table ${booking.table.number} at ${booking.time}"
      try
        retry(Schedule.exponentialBackoff(20.millis).maxRetries(2)):
          timeout(200.millis)(messenger.send(phone, text))
        s"$name: sent"
      catch
        case _: TimeoutException => s"$name: gateway didn't answer"
        case error: IOException  => s"$name: ${error.getMessage}"

@main def directConfirmations(): Unit =
  def numbersOf(name: String): Set[Phone] =
    tonight.filter(_.party.name == name).flatMap(_.party.phone).toSet

  val gateway = FlakyDirectGateway(numbersOf("Lindqvist"), numbersOf("Moreau"))
  val confirmed = tonight.filter(_.status == Status.Confirmed)
  val results = confirmed.mapPar(4)(booking => confirmDirect(gateway, booking))
  results.foreach(println)
  println(s"Handled ${results.size} bookings with ${gateway.totalCalls} calls to the gateway")
$ scala run . --main-class fullybooked.directConfirmations
Okonkwo: sent
Lindqvist: sent
Moreau: gateway didn't answer
Fitzgerald: no number, nothing sent
Handled 4 bookings with 7 calls to the gateway

The results are chapter 15’s, down to the seven calls. The gateway fails twice for the Lindqvists and hangs for the Moreaus, as before. confirmDirect again makes up to three attempts of a fifth of a second each, with a pause that starts at 20 milliseconds and doubles. This time retry and its schedule come from Ox, where chapter 15 had the retrying we wrote. Ox’s timeout works by interrupting the thread. Failures arrive as exceptions, so recoverWith has become try and catch, with finally available where chapter 15 used guarantee.

In directConfirmations, mapPar(4) sends up to four messages at a time and still returns the results in the order of the list. Two more tests for the end of DirectTests check the retry and the timeout. Each makes a set of one phone number by calling toSet on the Option that Phone.parse returns:

  test("a message that fails twice goes on the third call"):
    val number = Phone.parse("07700 900141").toSet
    val gateway = FlakyDirectGateway(failsTwice = number, hangs = Set())
    val booking = Booking(Party("Lindqvist", 2, number.headOption), Table(1, 2), eight, Status.Confirmed)
    assertEquals(confirmDirect(gateway, booking), "Lindqvist: sent")
    assertEquals(gateway.totalCalls, 3)

  test("a gateway that hangs is given up on"):
    val number = Phone.parse("07700 900141").toSet
    val gateway = FlakyDirectGateway(failsTwice = Set(), hangs = number)
    val booking = Booking(Party("Moreau", 2, number.headOption), Table(1, 2), eight, Status.Confirmed)
    assertEquals(confirmDirect(gateway, booking), "Moreau: gateway didn't answer")
    assertEquals(gateway.totalCalls, 3)

Interruption has the limit chapter 15 found with IO.blocking: a timeout can only stop code that responds to being interrupted. confirmDirect can be interrupted from outside as well, when the scope that mapPar opened is shutting down. Because InterruptedException is an Exception, a handler written as case error: Exception catches it and leaves a thread that has been told to stop still running. That’s why confirmDirect names the two types it means to handle.

17.5. Errors as values, without the for

A refused booking is still a Left in direct style. For chaining methods that return an Either, Ox has something shorter than a for. Add this to the end of direct.scala:

def bookDirect(bookings: DirectBookings, request: BookingRequest): Either[List[BookingError], Booking] =
  either:
    val (party, time) = validate(request).left.map(_.toList).ok()
    bookings.reserve(party, time).left.map(error => List(error)).ok()

Inside an either block, .ok() on a Right gives the value in it. On a Left it ends the whole block with that Left as the result. It reads like the for in chapter 9’s validateFast with the arrows taken out. Underneath, either uses boundary and break, which have been in Scala’s standard library since 3.3. They’re how direct-style code leaves a block early. A last test for DirectTests:

  test("an either block stops at the first Left"):
    val bookings = DirectBookings(List())
    val good = bookDirect(bookings, BookingRequest("Hughes", "4", "18:30", ""))
    assertEquals(good.map(_.table), Right(Table(3, 4)))
    val bad = bookDirect(bookings, BookingRequest("Olawale", "six", "19:15", ""))
    assertEquals(bad, Left(List(BookingError.BadPartySize("six"))))

17.6. What a signature no longer says

On chapter 14’s Bookings, reserve returns an IO. The signature says it does something to the outside world, and the compiler won’t let the result be used as if it were already a booking. Both the reserve on DirectBookings and decide return an Either. Nothing in the two types shows that one of them changes the list of bookings and the other has no effects.

The Scala team is working on ways to put that information back without the wrapper, through capabilities. The using Ox parameter is one already: a value passed as an ordinary parameter that the compiler keeps track of. Capture checking, an extension to the type system, goes further by following which capabilities a value holds on to. In Scala 3.9 it’s experimental and has to be switched on with a language import, so it isn’t something to build a service on yet.

17.7. Choosing

With Cats Effect a program is a value, and direct style has no counterpart to that. An IO can be handed to retrying, given a timeout or run beside another by code that knows nothing else about it. Its type tells every caller that it has effects. Cancelling works on an IO without the code inside having to cooperate, apart from cases like the IO.blocking one in chapter 15. Cats Effect also has the largest set of service libraries in Scala, including the ones Part IV’s service uses.

There’s very little to learn in direct style. The code reads like the Java it replaces and can call any blocking Java library as it stands. A stack trace shows the path the code took and a debugger steps through it line by line. Against that, the types say less, a val in the wrong place runs too early, and a thread only stops if the code running on it notices the interrupt.

I wouldn’t move a team that already runs Cats Effect services. A team arriving from Java with a new service to write will probably be productive sooner in direct style. Tapir, the library chapter 19 uses for HTTP, is SoftwareMill’s too and has a direct-style server that uses Ox. I’ve kept the rest of this book on Cats Effect because http4s and doobie, the libraries behind Part IV’s service, are built on it, and because the Scala services you might be asked to work on were probably written before virtual threads existed.

17.8. Exercises

Solutions are in the book’s downloadable code, under ch17/exercises.

  1. Write lookUpAllDirect(names: List[String]): List[String], in which each lookup sleeps for 200 milliseconds and all of them run at once. Test that three take well under 600 milliseconds. System.nanoTime() will do for the clock.

  2. Ox’s raceSuccess(a, b) runs two computations and returns the result of whichever succeeds first, interrupting the other. Use it to write firstToAnswerDirect(a: => String, b: => String): String.

  3. timeoutOption is timeout that returns an Option where timeout would throw. Write within[A](limit: FiniteDuration)(work: => A): Option[A].

  4. Write seatAllDirect(bookings: List[Booking], at: LocalTime): Either[BookingError, List[Booking]], the seatAll of chapter 9’s exercises, using an either block. .ok() works inside the lambda passed to map.

Part IV: A real service

18. From scala-cli to sbt

Fully Booked is thirty-nine Scala files in one directory. Each can see all the others and is compiled against the same list of libraries, so the booking rules from chapter 3 have Cats Effect, fs2 and Ox on their classpath for no reason. The scala command has no way to split a project into parts with their own libraries, and Part IV is about to add an HTTP layer and a database.

sbt is the build tool most Scala projects use. We’ll move the project to it and split the code into three parts with their own dependencies. Then we’ll make warnings into errors, set up formatting, and have a build server run the checks on every change. The build in this book is on sbt 2. Most of what’s been written about sbt describes version 1, so I’ve pointed out where the two differ in something you’re likely to read about.

18.1. sbt and a JDK

If you installed Scala with cs setup in chapter 1, you already have an sbt command. If not, the installation page at https://www.scala-sbt.org has the options. The sbt command is only a small launcher. Each project names the version of sbt it wants, and the launcher downloads that version the first time it’s needed.

sbt doesn’t fetch a JDK the way //> using jvm 25 did. It uses the JDK that JAVA_HOME points to, or the first java on your path if JAVA_HOME isn’t set, so check that the one it will find is version 25. If it isn’t, Coursier can supply JDK 25 for the current terminal on macOS or Linux:

$ eval "$(cs java --jvm 25 --env)"

To make the change permanent, run cs java --jvm 25 --setup. Running our programs needs Java 23 or later, because the JVM option we’ve been passing since chapter 6, --sun-misc-unsafe-memory-access=allow, doesn’t exist before that. An older JVM won’t start when it’s given an option it doesn’t know.

18.2. Three subprojects

sbt calls the parts of a build subprojects. Ours will be:

  • core, the code with no effects in it: the model, the rules, validation, the in-memory store and the command-line entry points from Parts I and II. It needs only Cats.

  • service, everything written against IO: configuration, the journal, concurrency, confirmations and the import. It needs core, Cats Effect, fs2, PureConfig and logging.

  • direct, chapter 17’s code. It needs core and Ox.

sbt expects the directory layout that Maven uses, with code under src/main/scala, tests under src/test/scala and other files under src/main/resources. From the top-level directory of the project, on macOS or Linux, these commands put everything where it belongs:

mkdir -p core/src/main/scala core/src/test/scala
mkdir -p service/src/main/scala service/src/main/resources service/src/test/scala
mkdir -p direct/src/main/scala direct/src/test/scala

mv direct.scala direct/src/main/scala/
mv direct.test.scala direct/src/test/scala/

mv bigfile.scala concurrency.scala config.scala confirmations.scala service/src/main/scala/
mv evening.scala importer.scala journal.scala service.scala service/src/main/scala/
mv concurrency.test.scala confirmations.test.scala service/src/test/scala/
mv effects.test.scala streams.test.scala service/src/test/scala/
mv resources/* service/src/main/resources/

mv *.test.scala core/src/test/scala/
rm project.scala
mv *.scala core/src/main/scala/
rmdir resources

On Windows, make the same moves in Explorer. The tests keep their .test.scala names, but under sbt a file is a test because of the directory it’s in. project.scala goes because sbt doesn’t read directives. nightly.csv stays where it is, because sbt runs our programs from the top-level directory.

18.3. The build definition

The version of sbt goes in project/build.properties, a file of a single line:

sbt.version=2.0.12

The build itself is described in build.sbt. It goes at the top level, beside nightly.csv, and not in the new project directory:

scalaVersion := "3.9.0"
organization := "com.fullybooked"
version := "0.1.0"

run / javaOptions += "--sun-misc-unsafe-memory-access=allow"

val munit = "org.scalameta" %% "munit" % "1.3.6" % Test

lazy val core = project
  .settings(
    libraryDependencies ++= Seq(
      "org.typelevel" %% "cats-core" % "2.13.0",
      munit
    )
  )

lazy val service = project
  .dependsOn(core)
  .settings(
    libraryDependencies ++= Seq(
      "org.typelevel" %% "cats-effect" % "3.7.1",
      "co.fs2" %% "fs2-io" % "3.14.0",
      "com.github.pureconfig" %% "pureconfig-core" % "0.17.11",
      "org.typelevel" %% "log4cats-slf4j" % "2.8.0",
      "ch.qos.logback" % "logback-classic" % "1.6.7",
      munit
    )
  )

lazy val direct = project
  .dependsOn(core)
  .settings(
    libraryDependencies ++= Seq(
      "com.softwaremill.ox" %% "core" % "1.1.0",
      munit
    )
  )

The scala command can write a file like this for you. The command is scala --power export --sbt . and it’s marked experimental. When I tried it on this project the result needed enough tidying that I wrote the file by hand.

A build.sbt is Scala with a few operators of sbt’s own. Things that sbt does, such as compile or test, are called tasks. Most of the file is a list of settings, the values those tasks work from. Each setting gives a value to a key: := sets it, += adds one element to a key that holds a list and ++= adds several. The dependencies are the ones project.scala listed, with the parts of each joined by %. The doubled %% does what :: did in a directive and picks the build of a library for our version of Scala. A % Test on the end keeps the library to the tests, as test.dep did.

Each lazy val ... = project declares a subproject in the directory of the same name. A lazy val is a val that isn’t worked out until it’s first used. sbt’s documentation uses one here so that subprojects can refer to one another in any order. The dependsOn(core) on service and direct puts core and its libraries on their classpaths. Neither of those two can see the other.

In sbt 2 a setting written at the top, outside any subproject, applies to every subproject. In sbt 1 such a setting didn’t reach the subprojects, which is why older builds write ThisBuild / scalaVersion. The slash in run / javaOptions narrows a key: these are the JVM options for running our programs, and not for the compiler or the tests.

Now compile:

$ sbt compile
[error] -- [E006] Not Found Error: .../fully-booked/direct/src/main/scala/direct.scala:23:20
[error] 23 |  def first = reply(ibrahim.name, reserve(store, ibrahim, eight))
[error]    |                    ^^^^^^^
[error]    |                    Not found: ibrahim
[error]    |
[error]    | longer explanation available when compiling with `-explain`
...
[error] 15 errors found

The first run is slow, because sbt downloads itself and compiles the build definition before it looks at our code. The fifteen errors that follow are all in direct. Its code uses ibrahim, castellanos, eight and regulars from concurrency.scala, and that file now lives in service, which direct isn’t allowed to see. Above the errors is a warning that begins multiple main classes detected. sbt prints it whenever it compiles a subproject with more than one entry point, and you can ignore it. The output shown in this chapter leaves it out.

Both service and direct use those four values, so they belong in core. Cut them from service/src/main/scala/concurrency.scala, from val eight to the end of regulars. Paste them at the end of core/src/main/scala/tonight.scala and compile again:

$ sbt compile
[success] elapsed time: 2 s

A file in core that imports cats.effect.IO won’t build any more. The booking rules can’t pick up a dependency on chapter 20’s database driver either, unless somebody edits build.sbt to allow it.

18.4. Running things

$ sbt projects
[info] In file:.../fully-booked/
[info] 	   core
[info] 	   direct
[info] 	 * fully-booked-root
[info] 	   service
[success] elapsed time: 2 s

There are four subprojects, because the directory itself is one. That fourth one is called the root, and ours has no code in it. A task run without a name in front goes to the root, which passes it on to the other three, so sbt test tests everything. sbt core/test tests only core. These are the last two of about thirty lines:

$ sbt core/test
[info] Passed: Total 35, Failed 0, Errors 0, Passed 35
[success] elapsed time: 2 s

Run the same command again:

$ sbt core/test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for core / Test / testQuick
[success] elapsed time: 2 s

In sbt 2, test only runs the tests that failed last time, have never run, or depend on code that has changed. sbt keeps the results on disk, so it still knows them after the machine restarts. To run every test regardless there’s testFull.

sbt has also been running as a server since the first sbt compile, with the build loaded and the compiler warm, and each later command connects to it. You stop it with sbt shutdown. That’s also the thing to try if sbt stops noticing an edit to build.sbt, or fails without saying why: a new server reads the file afresh and prints the real error. Typing sbt alone opens a prompt on the same server, where ~test re-runs the affected tests every time you save a file.

To run a program, name its subproject and its entry point. I’ve put the next command in quotes so that the shell hands it to sbt as one piece. sbt 2 insists on that for a list of commands, as in sbt "clean ; compile ; test", where sbt 1 accepted sbt clean compile test.

$ sbt "service/runMain fullybooked.ImportBookings"
18:30 table 3: Hughes, party of 4
19:00 table 1: Banerjee, party of 2
Line 3: 'six' isn't a number of people
19:30 table 4: Kowalczyk, party of 3
Line 5: we need a name for the booking
20:00 table 6: Mbeki, party of 8 (deposit taken)
Line 7: no table for 8 is free at 20:00
Line 8: '7pm' isn't a time, try 19:30; 'call the hotel' isn't a phone number
INFO  ImportBookings: Imported 4 bookings from nightly.csv
[success] elapsed time: 2 s

With no name, sbt service/run lists the entry points and asks which one you want. An sbt 1 build has a target directory in each subproject. In sbt 2, compiled classes go under target/out at the top level, with a subdirectory for each subproject.

18.5. Warnings as errors

In chapter 3 I said I tend to make all of my warnings hard errors. I do it because a warning that doesn’t stop the build is easy to scroll past. A project that lets warnings pile up can end with hundreds, and the one that matters is somewhere among them. Add this block to build.sbt after the first three lines. It asks the compiler for more warnings than it gives by default:

scalacOptions ++= Seq(
  "-deprecation",
  "-feature",
  "-Wunused:all",
  "-Wvalue-discard",
  "-Wnonunit-statement",
  "-Winfer-union"
)

The first two options, -deprecation and -feature, make the compiler spell out warnings it otherwise only counts. -Wunused:all reports imports, parameters and local values that nothing uses. -Wvalue-discard and -Wnonunit-statement catch a value being thrown away: one looks at places where a Unit was expected, the other at an expression on a line of its own in the middle of a block. -Winfer-union is behind the complaint we’re about to see. There’s also -Wall, which I don’t use on Scala 3.9. It includes a warning for every object other than a String inside an interpolated string, such as a LocalTime, and this project has about twenty of those.

The task Test/compile builds the tests as well as the main code. Start with core, which has one complaint:

$ sbt core/Test/compile
[warn] -- [E225] Type Warning: .../fully-booked/core/src/main/scala/main.scala:13:38
[warn] 13 |      val phone = booking.party.phone.getOrElse("none given")
[warn]    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[warn]    |A type argument was inferred to be union type fullybooked.Phone | String
[warn]    |This may indicate a programming error.

That’s the line in lookUp that chapter 8 left alone. It compiled because getOrElse was allowed to answer Phone | String, the union type chapter 10 explained. The compiler now says that a union it had to infer is probably a mistake. It’s right about this one, because the number prints without its space. Change the line to val phone = booking.party.phone.map(_.formatted).getOrElse("none given").

Now run sbt Test/compile, which compiles the other two subprojects. They have four warnings between them. The compiler only reports a warning when it compiles the file, so running the same command a second time prints none of them.

  • In effects.test.scala, the lambda error => IO.pure(0) never uses error. Write _ => IO.pure(0).

  • In manyThreads, in direct.scala, the lambda’s last line is seated.incrementAndGet(). That call returns an Int, which is thrown away because a Runnable returns nothing. Write seated.incrementAndGet(): Unit, which tells the compiler that the value is being dropped on purpose.

  • In direct.test.scala, the two fork: blocks in the scope test each return a Fork that nothing uses. Ox’s discard method says that’s intended. Change the file’s import of Ox to import ox.{discard, fork, mapPar, sleep, supervised}. Then put .discard on a line of its own after each block, at the same indentation as its fork:.

With those fixed, add "-Werror", as the first line of the list and compile once more:

$ sbt Test/compile
[success] elapsed time: 2 s

With -Werror in the list, a compiler warning now fails the build.

18.6. Formatting

Scala’s formatter is scalafmt. I’d sooner have a tool decide the layout than discuss it in code review. sbt runs scalafmt through a plugin, and plugins are declared in project/plugins.sbt:

addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.6.2")

The formatter’s own settings go in .scalafmt.conf at the top level, beside build.sbt:

version = 3.11.5
runner.dialect = scala3
maxColumn = 100
align.preset = none

The version is there so that everyone’s files come out the same, and the dialect so that scalafmt expects Scala 3 syntax. For the other two I’ve gone against the defaults. I’ve raised the line length from 80, and I’ve stopped scalafmt lining up the arrows of neighbouring cases, although the book’s code has lined them up until now. With alignment on, one long pattern pushes every arrow in the block across the page and a one-line change shows up in a diff as five.

$ sbt scalafmtCheckAll
[warn] scalafmt: .../fully-booked/core/src/main/scala/desk.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/display.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/errors.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/events.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/model.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/phone.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/request.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/core/src/main/scala/tonight.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/direct/src/main/scala/direct.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/direct/src/test/scala/direct.test.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/service/src/main/scala/concurrency.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/service/src/main/scala/confirmations.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/service/src/main/scala/service.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/service/src/test/scala/confirmations.test.scala isn't formatted properly!
[warn] scalafmt: .../fully-booked/service/src/test/scala/streams.test.scala isn't formatted properly!

The errors that follow the list of files aren’t shown here. scalafmtCheckAll changes nothing and fails if any file would be changed, which is the form a build server wants. The reformatting is done by sbt "scalafmtAll ; scalafmtSbt", where the second task is for build.sbt itself. Run that command and check again, this time with scalafmtSbtCheck for build.sbt as well:

$ sbt "scalafmtCheckAll ; scalafmtSbtCheck"
[success] elapsed time: 2 s

That rewrote fifteen of your files. Have a look at what changed in tonight.scala and events.scala. If the project is in Git, commit the reformatting on its own, so that later changes aren’t buried in it. Metals and IntelliJ both read .scalafmt.conf and can format a file when you save it.

18.7. A build server

A build server is another machine that runs the same checks on every change. This workflow for GitHub Actions goes in .github/workflows/ci.yml at the top level:

name: CI
on:
  pull_request:
  push:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the code
        uses: actions/checkout@v6
      - name: Set up Java
        uses: actions/setup-java@v5
        with:
          distribution: temurin
          java-version: 25
          cache: sbt
      - name: Set up sbt
        uses: sbt/setup-sbt@v1
      - name: Check formatting, compile and test
        run: sbt "scalafmtCheckAll ; scalafmtSbtCheck ; test"

The last step does the checking, with one sbt command made of the two formatting checks and the tests. The compiler options need no step of their own, since a warning already fails the compile that test depends on.

The setup-sbt action keeps sbt’s cache between runs, so with test in the workflow a test whose code hasn’t changed may not run again on the server either. Use testFull in the workflow if you want every test run on every change.

A .gitignore at the top level keeps the build’s output, and the files our programs write, out of the repository:

target/
.bsp/
.scala-build/
.metals/
.idea/
journal.log
big.csv

18.8. Exercises

There are no solutions to download for this chapter. The finished build is under ch18/project.

  1. Run one suite with sbt "core/test fullybooked.PhoneTests", twice. Then change an assertion in phone.test.scala so that it fails, and run sbt core/test twice more. Which tests ran each time?

  2. Add an unused import to a file in core, compile, and read what the compiler says. Then add import cats.effect.IO to the same file and compile again.

  3. sbt "show core / scalacOptions" prints the value of a setting, and sbt core/dependencyTree shows where each library on the classpath came from. Find out which version of the Scala library Cats was built against.

  4. Move bigfile.scala into a fourth subproject called tools that depends on service. What’s the least it needs in build.sbt?

  5. Change align.preset to more in .scalafmt.conf, reformat, and look at desk.scala. Put it back afterwards.

19. An HTTP API with Tapir and http4s

By the end of this chapter you’ll be able to book a table with curl. Three requests will work: one that lists tonight’s bookings, one that makes a booking and one that looks a booking up by name. A request that can’t be met gets an answer saying what’s wrong with it.

The server is http4s, an HTTP library built on Cats Effect and fs2 whose own server is called Ember. On top of it I use Tapir, where each endpoint is a value that describes its path, its input and its answers. Tapir produces both the http4s routes and the documentation from that value, so the paths and shapes in the documentation can’t fall behind the code.

19.1. One endpoint

Add four libraries to the service subproject’s list in build.sbt, after the line for Logback:

      "com.softwaremill.sttp.tapir" %% "tapir-http4s-server" % "1.14.1",
      "com.softwaremill.sttp.tapir" %% "tapir-json-circe" % "1.14.1",
      "com.softwaremill.sttp.tapir" %% "tapir-swagger-ui-bundle" % "1.14.1",
      "org.http4s" %% "http4s-ember-server" % "0.23.38",

The server will read its port from the configuration, so add port = 8080 as the last line of application.conf and give AppConfig a field for it:

case class AppConfig(
    restaurant: String,
    journal: String,
    storeDelay: FiniteDuration,
    port: Int
) derives ConfigReader

Then create api.scala in service/src/main/scala:

package fullybooked

import io.circe.Codec
import sttp.tapir.*
import sttp.tapir.json.circe.*

case class BookingView(name: String, size: Int, table: Int, time: String, status: String)
    derives Codec.AsObject,
      Schema

object BookingView:
  def of(booking: Booking): BookingView =
    BookingView(
      booking.party.name,
      booking.party.size,
      booking.table.number,
      booking.time.toString,
      describe(booking.status)
    )

val listEndpoint =
  endpoint.get
    .in("bookings")
    .out(jsonBody[List[BookingView]])
    .description("Every booking for tonight")

The class BookingView is a booking as the API shows it, in five plain fields. I keep these types apart from the model. A Booking holds an opaque Phone and a Status with data in two of its cases. The model should stay free to change without every client of the API noticing. The derives clause on BookingView asks circe, a JSON library, for a Codec.AsObject that can encode and decode the class. It asks Tapir for a Schema, which describes the class in the documentation.

The value listEndpoint describes one request, a GET to /bookings that’s answered with a list of BookingViews as JSON. I’ve left the compiler to work out its type, because it’s long. Among other things the type records what the endpoint takes in, what it answers with when it fails and what it answers with when it succeeds. Here those are nothing, nothing and the list.

The code that produces the list goes in a second file, routes.scala, beside the first:

package fullybooked

import cats.effect.IO
import sttp.capabilities.fs2.Fs2Streams
import sttp.tapir.server.ServerEndpoint

def bookingRoutes(bookings: Bookings): List[ServerEndpoint[Fs2Streams[IO], IO]] =
  List(
    listEndpoint.serverLogicSuccess(_ => bookings.all.map(_.map(BookingView.of)))
  )

Calling serverLogicSuccess on an endpoint gives it a function that takes the endpoint’s input and returns an IO of its output. The result is a ServerEndpoint: the description together with the code that answers the request. Its two type parameters say that our endpoints may use fs2’s streams, though they don’t, and that they run in IO. The Bookings passed in is the class from chapter 14 that keeps its list in a Ref.

The third file, server.scala, goes in the same directory:

package fullybooked

import cats.effect.{IO, IOApp}
import com.comcast.ip4s.{ipv4, Port}
import org.http4s.ember.server.EmberServerBuilder
import org.http4s.server.middleware.Logger
import sttp.tapir.server.http4s.Http4sServerInterpreter
import sttp.tapir.swagger.bundle.SwaggerInterpreter

object Server extends IOApp.Simple:
  def run: IO[Unit] =
    for
      config <- loadConfig
      port <- IO.fromOption(Port.fromInt(config.port))(
        IllegalArgumentException(s"${config.port} isn't a port")
      )
      bookings <- Bookings.starting(List())
      routes = bookingRoutes(bookings)
      docs = SwaggerInterpreter().fromServerEndpoints[IO](routes, config.restaurant, "0.1.0")
      app = Http4sServerInterpreter[IO]().toRoutes(routes ++ docs).orNotFound
      _ <- EmberServerBuilder
        .default[IO]
        .withHost(ipv4"0.0.0.0")
        .withPort(port)
        .withHttpApp(Logger.httpApp(logHeaders = false, logBody = false)(app))
        .build
        .useForever
    yield ()

The run method loads the configuration, makes an empty Bookings and gets the routes for it. Tapir’s SwaggerInterpreter writes documentation for those routes, which comes back as more endpoints to serve. Http4sServerInterpreter then turns the lot into an http4s application, with orNotFound answering 404 for any other path.

Ember serves that application on every network interface, at the port from the configuration. The ipv4 in front of "0.0.0.0" is a string interpolator like s. It checks the address when the code compiles, so a mistyped address is a compile error. A port that arrives as an Int at run time can’t be checked that way. Port.fromInt returns an Option[Port], which is None for a number that can’t be a port. IO.fromOption turns a None into a failed IO, so a bad port stops the service at start-up. Before the application is handed to Ember it’s wrapped in Logger.httpApp to log each request and response. The server that build returns is a Resource, and useForever keeps it open until the program is stopped.

Start the server:

$ sbt "service/runMain fullybooked.Server"
INFO  EmberServerBuilderCompanionPlatform: Ember-Server service bound to address: [::]:8080
INFO  Logger: HTTP/1.1 GET /bookings
INFO  Logger: HTTP/1.1 200 OK

The [::] in the first log line is how Java reports a socket that listens on every interface. The lines after it appear as requests arrive. In a second terminal, ask for the bookings:

$ curl localhost:8080/bookings
[]

Then open http://localhost:8080/docs in a browser. The page lists the endpoint. Click on it for its description, the shape of a BookingView and a button that sends the request. Behind the page is a document in the OpenAPI format that other tools can read:

$ curl localhost:8080/docs/docs.yaml
openapi: 3.1.0
info:
  title: Fully Booked
  version: 0.1.0
paths:
  /bookings:
    get:
      description: Every booking for tonight
      operationId: getBookings
      responses:
        '200':
          description: ''
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/BookingView'
...

19.2. Taking a booking

Stop the server with Ctrl+C. sbt may print an exception about the way the program ended. You can ignore it. The other two endpoints need a status code in their types, so add import sttp.model.StatusCode under the import of Codec in api.scala. Then add this to the end of the file:

case class NewBooking(name: String, size: Int, time: String, phone: Option[String])
    derives Codec.AsObject,
      Schema

case class Problems(problems: List[String]) derives Codec.AsObject, Schema

type Refusal = (StatusCode, Problems)

val createEndpoint =
  endpoint.post
    .in("bookings")
    .in(jsonBody[NewBooking])
    .out(statusCode(StatusCode.Created))
    .out(jsonBody[BookingView])
    .errorOut(statusCode)
    .errorOut(jsonBody[Problems])
    .description("Book a table")

val findEndpoint =
  endpoint.get
    .in("bookings" / path[String]("name"))
    .out(jsonBody[BookingView])
    .errorOut(statusCode)
    .errorOut(jsonBody[Problems])
    .description("The booking under one name")

A client sends a NewBooking, in which size is a number and phone can be left out. When a request fails, the client gets back Problems, a list of messages. A Refusal is a tuple of a status code and the Problems to send with it.

In the two endpoints, each in adds an input, which can be a piece of the path or the body. "bookings" / path[String]("name") is two pieces of path, the second of which is handed to our code as a String. Among the outputs, statusCode(StatusCode.Created) fixes the status of a successful POST at 201. The two errorOut lines say that a failure is a status code followed by Problems as JSON.

Replace everything in routes.scala with the logic for all three:

package fullybooked

import cats.effect.IO
import sttp.capabilities.fs2.Fs2Streams
import sttp.model.StatusCode
import sttp.tapir.server.ServerEndpoint

def refusal(status: StatusCode, problems: List[String]): Refusal = (status, Problems(problems))

def create(bookings: Bookings, request: NewBooking): IO[Either[Refusal, BookingView]] =
  val asText =
    BookingRequest(request.name, request.size.toString, request.time, request.phone.getOrElse(""))
  validate(asText) match
    case Left(errors) =>
      IO.pure(Left(refusal(StatusCode.BadRequest, errors.toList.map(explain))))
    case Right((party, time)) =>
      bookings
        .reserve(party, time)
        .map:
          case Right(booking) => Right(BookingView.of(booking))
          case Left(error) => Left(refusal(StatusCode.Conflict, List(explain(error))))

def find(bookings: Bookings, name: String): IO[Either[Refusal, BookingView]] =
  bookings.all.map: all =>
    findBooking(name, all)
      .map(BookingView.of)
      .toRight(refusal(StatusCode.NotFound, List(s"no booking under $name")))

def bookingRoutes(bookings: Bookings): List[ServerEndpoint[Fs2Streams[IO], IO]] =
  List(
    listEndpoint.serverLogicSuccess(_ => bookings.all.map(_.map(BookingView.of))),
    createEndpoint.serverLogic(request => create(bookings, request)),
    findEndpoint.serverLogic(name => find(bookings, name))
  )

The function create turns the NewBooking into the four strings of a BookingRequest and hands them to validate, the function that checked the command line’s four strings in chapter 9. A request that fails validation is the client’s mistake and gets 400 with every problem listed. One that’s well formed but refused by reserve, usually because the tables are taken, gets 409. In find, a name with no booking gets 404.

Where serverLogicSuccess took a function with no error value to return, serverLogic wants one that returns an IO of an Either, with the endpoint’s error type on the left. create and find are ordinary functions of that shape, so they can be tested with no server running. The test file below has a helper called answers that sends a list of requests through create one after another. Its traverse works on IO as it did on Either in chapter 11. It keeps the name from each success and the status code from each refusal. Create api.test.scala in service/src/test/scala:

package fullybooked

import cats.effect.unsafe.implicits.global
import cats.syntax.all.*
import sttp.model.StatusCode

class ApiTests extends munit.FunSuite:
  def answers(requests: List[NewBooking]): List[Either[StatusCode, String]] =
    val program =
      for
        bookings <- Bookings.starting(regulars)
        results <- requests.traverse(request => create(bookings, request))
      yield results.map(result => result.left.map(refused => refused(0)).map(view => view.name))
    program.unsafeRunSync()

  test("a good request is booked"):
    assertEquals(answers(List(NewBooking("Ibrahim", 7, "20:00", None))), List(Right("Ibrahim")))

  test("a request that fails validation is a 400 listing every problem"):
    val bad = NewBooking("", 7, "8pm", Some("ask for Dev"))
    assertEquals(answers(List(bad)), List(Left(StatusCode.BadRequest)))
    val program = Bookings.starting(regulars).flatMap(bookings => create(bookings, bad))
    assertEquals(program.unsafeRunSync().left.map(refused => refused(1).problems.size), Left(3))

  test("the second party after the last table gets a 409"):
    val both = List(
      NewBooking("Ibrahim", 7, "20:00", None),
      NewBooking("Castellanos", 8, "20:00", None)
    )
    assertEquals(answers(both), List(Right("Ibrahim"), Left(StatusCode.Conflict)))

  test("a name with no booking is a 404"):
    val program = Bookings.starting(regulars).flatMap(bookings => find(bookings, "Smith"))
    assertEquals(program.unsafeRunSync().left.map(refused => refused(0)), Left(StatusCode.NotFound))
$ sbt "service/test fullybooked.ApiTests"
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] elapsed time: 2 s

Start the server again and book a table. If typing JSON into a terminal doesn’t appeal, or you’re on Windows where the quoting is different, the page at /docs will send the same requests.

$ curl -d '{"name":"Hughes","size":4,"time":"18:30"}' -H 'Content-Type: application/json' localhost:8080/bookings
{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}

The -d option supplies a body and makes the request a POST. Next, send a request with two things wrong, adding -i to show the status line and headers of the answer. I’ve left the Date header out of the listing:

$ curl -i -d '{"name":"","size":4,"time":"7pm"}' -H 'Content-Type: application/json' localhost:8080/bookings
HTTP/1.1 400 Bad Request
Connection: keep-alive
Content-Type: application/json
Content-Length: 79

{"problems":["we need a name for the booking","'7pm' isn't a time, try 19:30"]}

Book the Mbekis in for eight people at 20:00 in the same way as the Hugheses. Then try the Tanakas, who want the same table. -w '\n%{http_code}\n' prints the status on a line of its own after the body:

$ curl -d '{"name":"Tanaka","size":8,"time":"20:00"}' -H 'Content-Type: application/json' -w '\n%{http_code}\n' localhost:8080/bookings
{"problems":["no table for 8 is free at 20:00"]}
409

The Tanakas get a 409 because the Mbekis have the last table for eight. If the two requests had arrived at the same moment, one party would still have got the table and the other a 409. The check and the save in Bookings.reserve are one atomic step.

Look a booking up under a name that has one, and then under a name that hasn’t:

$ curl -w '\n%{http_code}\n' localhost:8080/bookings/Mbeki
{"name":"Mbeki","size":8,"table":6,"time":"20:00","status":"confirmed"}
200
$ curl -w '\n%{http_code}\n' localhost:8080/bookings/Smith
{"problems":["no booking under Smith"]}
404

Send a body in which the size isn’t a number:

$ curl -d '{"name":"Tanaka","size":"eight","time":"20:00"}' -H 'Content-Type: application/json' -w '\n%{http_code}\n' localhost:8080/bookings
Invalid value for: body (Int at 'size')
400

The body isn’t a NewBooking, since "eight" isn’t a number, so Tapir answers 400 by itself. The message is Tapir’s and it’s plain text, where our own failures are JSON. If a client ever reports an error message that you can’t find in your own code, this is one place to look. Tapir lets you replace its answer with one of your own, which I’d do for a public API.

19.3. Middleware

http4s calls Logger.httpApp a middleware: a function that takes an application and returns another, which does something before or after it passes each request on. Because the logger wraps the whole application, the log shows the requests for the documentation and the ones that ended in 404, as well as those for bookings.

I use a middleware when something should happen to every request whatever the endpoint. http4s has them for timing requests out and for CORS headers, among others. Chapter 22 adds two when it gets the service ready to deploy.

19.4. Exercises

Solutions are in the book’s downloadable code, under ch19/exercises.

  1. Add DELETE /bookings/{name}, which removes a booking. It should answer 204 with no body, or 404 if there was no booking under that name. Bookings will need a cancel method, and endpoint.delete starts the description.

  2. Let GET /bookings?size=4 list only the parties of four or more. query[Option[Int]]("size") describes an optional query parameter.

  3. A party of nine gets 409, as though the restaurant were full. It can never be seated, so 400 would be a better answer. Write statusFor(error: BookingError): StatusCode and use it in create.

  4. Add GET /tables, which lists the restaurant’s tables. Table lives in core, which knows nothing about JSON, so it will need a view class of its own.

  5. Open /docs after each of the exercises above and find what changed.

20. Postgres with doobie

Stop chapter 19’s server, start it again and ask for the bookings. The Hugheses and the Mbekis have gone. Their bookings only ever existed in a Ref, which lasts as long as the process that made it.

We’ll keep the bookings in PostgreSQL, through doobie, a library that runs SQL from Cats Effect. It isn’t an ORM of the kind Hibernate is. You write the queries yourself, in SQL, and doobie deals with connections, transactions and turning rows into values. I prefer that, because I can see exactly the SQL that will run.

20.1. A database to talk to

The simplest way to get a Postgres for development is Docker, which chapter 22 needs in any case. If you don’t have it, https://docs.docker.com/get-started/get-docker/ has the installers. This command starts a Postgres in the background:

$ docker run --name fully-booked-db -e POSTGRES_PASSWORD=table-for-two \
    -e POSTGRES_DB=fullybooked -p 5432:5432 -d postgres:18

To turn it off and on, use docker stop fully-booked-db and docker start fully-booked-db. The data survives both. If you already run Postgres on your machine, create a database called fullybooked in it and use your own user and password below.

Add four lines to the service list in build.sbt, after the one for http4s-ember-server:

      "org.tpolecat" %% "doobie-core" % "1.0.0-RC12",
      "org.tpolecat" %% "doobie-hikari" % "1.0.0-RC12",
      "org.tpolecat" %% "doobie-postgres" % "1.0.0-RC12",
      "org.flywaydb" % "flyway-database-postgresql" % "13.12.2",

The service has to know where the database is. Add a block to the end of application.conf:

database {
  url = "jdbc:postgresql://localhost:5432/fullybooked"
  url = ${?DATABASE_URL}
  user = "postgres"
  password = "table-for-two"
}

The block sets url twice. The second setting only applies when the environment variable DATABASE_URL exists, so the file works as it is on your machine and can be pointed somewhere else without being edited. Do the same for the port while you’re there, by adding port = ${?PORT} under the line that sets it to 8080. A password in a file is fine for a database that lives on your laptop. Chapter 22 moves it out of the file. In config.scala, add a class for the block and a field to AppConfig:

case class DatabaseConfig(url: String, user: String, password: String) derives ConfigReader

case class AppConfig(
    restaurant: String,
    journal: String,
    storeDelay: FiniteDuration,
    port: Int,
    database: DatabaseConfig
) derives ConfigReader

20.2. Migrations

We’ll create the table, and change it later as the service grows, with migrations. These are numbered SQL files that a library called Flyway applies in order. Create the directories db/migration under service/src/main/resources and put the first migration in a file called V1__bookings.sql, with two underscores:

CREATE TABLE bookings (
  id            BIGSERIAL PRIMARY KEY,
  name          TEXT NOT NULL,
  party_size    INT  NOT NULL,
  phone         TEXT,
  table_number  INT  NOT NULL,
  starts        TIME NOT NULL,
  ends          TIME NOT NULL,
  status        TEXT NOT NULL,
  seated_at     TIME,
  cancel_reason TEXT
);

A booking’s status takes three columns, because two of the statuses carry data. The ends column, the time the party is due to leave, is there for the second migration.

Now create database.scala in service/src/main/scala:

package fullybooked

import cats.effect.{IO, Resource}
import com.zaxxer.hikari.HikariConfig
import doobie.Transactor
import doobie.hikari.HikariTransactor
import org.flywaydb.core.Flyway

def migrate(config: DatabaseConfig): IO[Int] =
  IO.blocking:
    Flyway
      .configure()
      .dataSource(config.url, config.user, config.password)
      .load()
      .migrate()
      .migrationsExecuted

def transactor(config: DatabaseConfig): Resource[IO, Transactor[IO]] =
  val pool = HikariConfig()
  pool.setJdbcUrl(config.url)
  pool.setUsername(config.user)
  pool.setPassword(config.password)
  pool.setMaximumPoolSize(8)
  HikariTransactor.fromHikariConfig[IO](pool)

The first function, migrate, runs every migration the database hasn’t seen. Flyway keeps a table of its own in the database listing the ones it has applied, which makes it safe to call migrate on every start. It refuses to run if a migration in that table has been edited since. A change to the schema goes in a new file with the next number.

The second function returns a Transactor, the thing doobie runs SQL through, together with a pool of connections. Opening a connection takes long enough that a service keeps a few open and shares them out among the requests that need one. This pool has eight, so at most eight pieces of database work are in progress at once and a ninth waits its turn. The pool itself is HikariCP, a Java library. The Transactor is wrapped in a Resource because the pool has to be closed.

Flyway and Hikari both log a good deal at INFO. Add two lines to logback.xml, above the <root> element, to turn them down:

  <logger name="org.flywaydb" level="WARN"/>
  <logger name="com.zaxxer.hikari" level="WARN"/>

20.3. Rows and bookings

The rest of this chapter’s Scala files go in service/src/main/scala. A row in the table is ten columns of text, numbers and times. A Booking has a Party with an opaque Phone, a Table and a Status that’s an enum. I do the translating in plain functions that can be tested without a database. Create rows.scala:

package fullybooked

import doobie.Meta
import java.time.LocalTime

given Meta[Phone] =
  Meta[String].tiemap(text => Phone.parse(text).toRight(s"not a phone number: $text"))(_.toString)

case class BookingRow(
    name: String,
    size: Int,
    phone: Option[Phone],
    table: Int,
    starts: LocalTime,
    status: String,
    seatedAt: Option[LocalTime],
    cancelReason: Option[String]
)

def toRow(booking: Booking): BookingRow =
  val (seatedAt, cancelReason) = booking.status match
    case Status.Seated(at) => (Some(at), None)
    case Status.Cancelled(reason) => (None, Some(reason))
    case _ => (None, None)
  BookingRow(
    booking.party.name,
    booking.party.size,
    booking.party.phone,
    booking.table.number,
    booking.time,
    booking.status.productPrefix,
    seatedAt,
    cancelReason
  )

def toBooking(row: BookingRow): Either[String, Booking] =
  for
    table <- Table.all.find(_.number == row.table).toRight(s"no table numbered ${row.table}")
    status <- (row.status, row.seatedAt, row.cancelReason) match
      case ("Requested", _, _) => Right(Status.Requested)
      case ("Confirmed", _, _) => Right(Status.Confirmed)
      case ("Completed", _, _) => Right(Status.Completed)
      case ("NoShow", _, _) => Right(Status.NoShow)
      case ("Seated", Some(at), _) => Right(Status.Seated(at))
      case ("Cancelled", _, Some(reason)) => Right(Status.Cancelled(reason))
      case _ => Left(s"can't read the status of ${row.name}: ${row.status}")
  yield Booking(Party(row.name, row.size, row.phone), table, row.starts, status)

There’s a field in BookingRow for each column we read. doobie already knows how to fetch most of the fields' types, and the given Meta[Phone] teaches it Phone. tiemap builds a Meta[Phone] from the one for String and two functions, one for each direction. The function for reading returns an Either, since the column might hold something Phone.parse rejects.

Going from a booking to a row can’t fail, so toRow returns a plain BookingRow. Coming back, toBooking returns an Either with a message on the left, because a row could hold a table number we don’t have or a status we’ve never heard of. The status is stored as its productPrefix, the name of an enum case as a string: "Seated" for Status.Seated(at).

Create database.test.scala in service/src/test/scala and run it with sbt "service/test fullybooked.RowTests":

package fullybooked

import java.time.LocalTime

class RowTests extends munit.FunSuite:
  val phone = Phone.parse("07700 900141")
  val booking = Booking(Party("Okonkwo", 5, phone), Table(5, 6), LocalTime.of(19, 30))

  test("a booking survives the trip to a row and back in every status"):
    val statuses = List(
      Status.Requested,
      Status.Confirmed,
      Status.Completed,
      Status.NoShow,
      Status.Seated(LocalTime.of(19, 35)),
      Status.Cancelled("flu")
    )
    statuses.foreach: status =>
      val original = booking.copy(status = status)
      assertEquals(toBooking(toRow(original)), Right(original))

  test("a row that makes no sense is refused"):
    val row = toRow(booking)
    assert(toBooking(row.copy(table = 12)).isLeft)
    assert(toBooking(row.copy(status = "Seated", seatedAt = None)).isLeft)
$ sbt "service/test fullybooked.RowTests"
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] elapsed time: 2 s

20.4. Queries and transactions

The routes from chapter 19 are written against Bookings, a class with a Ref inside. They shouldn’t have to care where bookings are kept, so Bookings becomes a trait. In concurrency.scala, replace the class and its companion object with this:

trait Bookings:
  def all: IO[List[Booking]]
  def reserve(party: Party, time: LocalTime): IO[Either[BookingError, Booking]]

class InMemoryBookings(ref: Ref[IO, List[Booking]]) extends Bookings:
  def all: IO[List[Booking]] = ref.get

  def reserve(party: Party, time: LocalTime): IO[Either[BookingError, Booking]] =
    ref.modify: bookings =>
      decide(bookings, party, time) match
        case Right(booking) => (bookings :+ booking, Right(booking))
        case Left(error) => (bookings, Left(error))

object Bookings:
  def starting(bookings: List[Booking]): IO[Bookings] =
    Ref.of[IO, List[Booking]](bookings).map(ref => InMemoryBookings(ref))

Bookings.starting still exists and now makes an InMemoryBookings, so the code that called it compiles as before. The second implementation goes in a new file, postgres.scala:

package fullybooked

import cats.effect.IO
import cats.syntax.all.*
import doobie.*
import doobie.implicits.*
import doobie.postgres.implicits.*
import java.time.LocalTime

val selectBookings: Query0[BookingRow] =
  sql"""SELECT name, party_size, phone, table_number, starts, status, seated_at, cancel_reason
        FROM bookings
        ORDER BY starts, id""".query[BookingRow]

def insert(booking: Booking): Update0 =
  val row = toRow(booking)
  sql"""INSERT INTO bookings
          (name, party_size, phone, table_number, starts, ends, status, seated_at, cancel_reason)
        VALUES
          (${row.name}, ${row.size}, ${row.phone}, ${row.table}, ${row.starts}, ${booking.leaveBy},
           ${row.status}, ${row.seatedAt}, ${row.cancelReason})""".update

val loadBookings: ConnectionIO[List[Booking]] =
  selectBookings
    .to[List]
    .flatMap: rows =>
      rows.traverse(row => toBooking(row).leftMap(IllegalStateException(_)).liftTo[ConnectionIO])

class PostgresBookings(xa: Transactor[IO]) extends Bookings:
  def all: IO[List[Booking]] = loadBookings.transact(xa)

  def reserve(party: Party, time: LocalTime): IO[Either[BookingError, Booking]] =
    val attempt: ConnectionIO[Either[BookingError, Booking]] =
      loadBookings.flatMap: bookings =>
        decide(bookings, party, time) match
          case Left(error) => Left(error).pure[ConnectionIO]
          case Right(booking) => insert(booking).run.map(_ => Right(booking))
    attempt
      .transact(xa)

The queries are written with doobie’s sql interpolator, inside the triple quotes that Scala uses for a string that runs over several lines. A value interpolated with $ is sent to the database separately from the SQL text, as a parameter, so a name a customer typed can’t change the query. The columns that come back are matched to the fields of BookingRow by position. The last doobie import is there so that a LocalTime can be one of them.

Nothing in the first three definitions talks to a database. A Query0 or an Update0 describes a statement. Calling to[List] on a query, or run on an update, gives a ConnectionIO, a description of work to be done on one connection. A ConnectionIO has flatMap, as an IO does, and loadBookings uses it to follow the query with toBooking on each row. A row that can’t be read fails the whole ConnectionIO, because a table we can’t make sense of is a fault in the system and chapter 15 said those belong in a failed effect. leftMap, Cats’s name for left.map, turns the message into an exception. liftTo[ConnectionIO] then turns the Either into a ConnectionIO: a Right becomes one that succeeds and a Left one that fails with the exception.

A ConnectionIO becomes an IO at transact(xa), which borrows a connection from the pool, runs everything in one transaction and commits, or rolls back if any part fails. So in reserve, reading the bookings, deciding and inserting are one transaction. The .pure[ConnectionIO] in reserve wraps a plain value in a ConnectionIO, as IO.pure does for IO. There’s no IO.blocking in this file because doobie itself moves the blocking JDBC calls off Cats Effect’s main pool of threads.

20.5. The last table, in a transaction

One transaction sounds like the atomic step from chapter 14. Create lasttable.scala to find out:

package fullybooked

import cats.effect.{IO, IOApp}
import cats.syntax.all.*
import doobie.implicits.*

object LastTableDb extends IOApp.Simple:
  def run: IO[Unit] =
    for
      config <- loadConfig
      _ <- migrate(config.database)
      holders <- transactor(config.database).use: xa =>
        val bookings = PostgresBookings(xa)
        val parties = List.tabulate(20)(n => Party(s"Party $n", 7))
        for
          _ <- sql"TRUNCATE bookings".update.run.transact(xa)
          _ <- regulars.traverse_(booking => insert(booking).run).transact(xa)
          _ <- parties.parTraverse_(party => bookings.reserve(party, eight))
          stored <- bookings.all
        yield stored.count(_.table.number == 6)
      _ <- IO.println("20 parties asked for table 6 at eight")
      _ <- IO.println(if holders == 1 then "One of them has it" else "More than one of them has it")
    yield ()

The program empties the bookings table in the database and books the regulars into every table but number 6. Then it sends twenty parties of seven after that one, all at once. At the end it counts the bookings for table 6.

$ sbt "service/runMain fullybooked.LastTableDb"
20 parties asked for table 6 at eight
More than one of them has it

On my machine a few of the twenty end up with table 6. A transaction guarantees that its own statements happen together or not at all. It doesn’t, at Postgres’s default setting, stop a second transaction reading the same rows in the meantime. Two of our transactions each read the bookings, saw nobody at table 6, decided it was free and inserted.

Chapter 14’s Ref and semaphore each live in one process, so neither would help once two copies of the service are running. I’d rather the rule were in the database, which is the one place every writer has to go through, whether that’s a second copy of the service or somebody fixing a booking by hand. Add a second migration, V2__one_party_per_table.sql, beside the first:

TRUNCATE bookings;

CREATE EXTENSION btree_gist;
CREATE TYPE timerange AS RANGE (subtype = time);

ALTER TABLE bookings
  ADD CONSTRAINT one_party_per_table
  EXCLUDE USING gist (table_number WITH =, timerange(starts, ends) WITH &&)
  WHERE (status IN ('Confirmed', 'Seated'));

The constraint says that no two rows may have the same table number and overlapping times, counting only the bookings that hold a table. The times are the range from starts to ends. Postgres has no range type for times of day built in, so the CREATE TYPE line makes one. The btree_gist extension lets a plain number such as table_number take part in the constraint’s gist index. The TRUNCATE is there because the bookings table already contains a double booking, and Postgres won’t add the constraint to a table that breaks it. A migration on real data would have to repair the rows first.

With the constraint in place, one of two clashing inserts fails with an exception. Catching it and trying again isn’t enough when many parties clash at once. An insert that clashes with another transaction’s insert waits for that transaction to finish. When several are waiting for each other, Postgres calls it a deadlock and fails one of them. If that reservation tries again at once, it joins the others and waits again. With twenty parties, LastTableDb can go on like that and never finish. We’ll have Postgres skip a clashing row instead. In postgres.scala, change the end of the SQL in insert so that the function reads:

def insert(booking: Booking): Update0 =
  val row = toRow(booking)
  sql"""INSERT INTO bookings
          (name, party_size, phone, table_number, starts, ends, status, seated_at, cancel_reason)
        VALUES
          (${row.name}, ${row.size}, ${row.phone}, ${row.table}, ${row.starts}, ${booking.leaveBy},
           ${row.status}, ${row.seatedAt}, ${row.cancelReason})
        ON CONFLICT DO NOTHING""".update

With ON CONFLICT DO NOTHING, an insert that clashes with a row already in the table adds no row and raises no error. If the other row’s transaction is still in progress, Postgres waits for it, then skips our row if it commits and inserts ours if it doesn’t. Running an Update0 gives the number of rows it changed: for this insert, 1 or 0. Replace reserve with a version that looks at that number:

  def reserve(party: Party, time: LocalTime): IO[Either[BookingError, Booking]] =
    val attempt: ConnectionIO[Option[Either[BookingError, Booking]]] =
      loadBookings.flatMap: bookings =>
        decide(bookings, party, time) match
          case Left(error) => Some(Left(error)).pure[ConnectionIO]
          case Right(booking) =>
            insert(booking).run.map(rows => if rows == 1 then Some(Right(booking)) else None)
    def upTo(tries: Int): IO[Either[BookingError, Booking]] =
      attempt
        .transact(xa)
        .flatMap:
          case Some(answer) => IO.pure(answer)
          case None if tries > 1 => upTo(tries - 1)
          case None => IO.raiseError(IllegalStateException(s"ten clashes for ${party.name}"))
    upTo(10)

A transaction that inserted nothing answers None, and upTo then tries the whole reservation again in a new transaction. The booking that won has been committed by then, so loadBookings returns it and decide picks another table or says there’s none. Each lost attempt means one more of the six tables is taken, so ten attempts are more than a reservation can need. If all ten are lost, the database is refusing a table that decide thinks is free, perhaps because a row was changed by hand. When that happens, reserve gives up with a failed IO and the service answers 500.

$ sbt "service/runMain fullybooked.LastTableDb"
20 parties asked for table 6 at eight
One of them has it

20.6. Serving from the database

The server now has to run the migrations, open the pool and hand the routes a PostgresBookings. Replace server.scala:

package fullybooked

import cats.effect.{IO, IOApp}
import com.comcast.ip4s.{ipv4, Port}
import org.http4s.ember.server.EmberServerBuilder
import org.http4s.server.middleware
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import sttp.tapir.server.http4s.Http4sServerInterpreter
import sttp.tapir.swagger.bundle.SwaggerInterpreter

object Server extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def run: IO[Unit] =
    for
      config <- loadConfig
      applied <- migrate(config.database)
      _ <- logger.info(s"$applied migrations applied")
      port <- portFrom(config)
      _ <- transactor(config.database).use(xa => serve(config, port, PostgresBookings(xa)))
    yield ()

  def portFrom(config: AppConfig): IO[Port] =
    IO.fromOption(Port.fromInt(config.port))(
      IllegalArgumentException(s"${config.port} isn't a port")
    )

  def serve(config: AppConfig, port: Port, bookings: Bookings): IO[Unit] =
    val routes = bookingRoutes(bookings)
    val docs = SwaggerInterpreter().fromServerEndpoints[IO](routes, config.restaurant, "0.1.0")
    val app = Http4sServerInterpreter[IO]().toRoutes(routes ++ docs).orNotFound
    EmberServerBuilder
      .default[IO]
      .withHost(ipv4"0.0.0.0")
      .withPort(port)
      .withHttpApp(middleware.Logger.httpApp(logHeaders = false, logBody = false)(app))
      .build
      .useForever

http4s and log4cats both have something called Logger, so the middleware is written middleware.Logger here. The use around serve keeps the pool open for as long as the server runs. Empty the bookings table, so that the regulars don’t get in the way:

$ docker exec fully-booked-db psql -U postgres fullybooked -c 'TRUNCATE bookings'

Start the server with sbt "service/runMain fullybooked.Server" and make a booking:

$ curl -d '{"name":"Hughes","size":4,"time":"18:30","phone":"07700 900201"}' -H 'Content-Type: application/json' localhost:8080/bookings
{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}

Now stop the server with Ctrl+C and start it again:

$ sbt "service/runMain fullybooked.Server"
INFO  Server: 0 migrations applied
INFO  EmberServerBuilderCompanionPlatform: Ember-Server service bound to address: [::]:8080

There were no migrations to apply, since LastTableDb had already run both. The booking is still there:

$ curl localhost:8080/bookings
[{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}]

You can see it from outside the service as well:

$ docker exec fully-booked-db psql -U postgres fullybooked -c 'SELECT name, table_number, starts, ends, status FROM bookings'
  name  | table_number |  starts  |   ends   |  status
--------+--------------+----------+----------+-----------
 Hughes |            3 | 18:30:00 | 20:00:00 | Confirmed
(1 row)

20.7. Exercises

Solutions are in the book’s downloadable code, under ch20/exercises. ExerciseRun there runs all four against the database.

  1. find in routes.scala loads every booking and searches the list. Write selectByName(name: String): Query0[BookingRow], which has the database do the search.

  2. Write coversByTable: Query0[(Int, Int)], the number of people booked at each table, counting only bookings that hold one. sum(party_size)::int gives a column doobie can read as an Int.

  3. Write cancel(name: String, reason: String): Update0, which marks a requested or confirmed booking as cancelled. run on an Update0 gives the number of rows changed. What should the service answer when that’s 0?

  4. Write moveToLater(name: String, minutes: Int): ConnectionIO[Int], which moves a booking’s start and end later by that many minutes, and fails when no row was changed. In SQL, starts + ($minutes * interval '1 minute') adds minutes to a time. Fragments made with sql can be joined with ++.

21. Testing the service

Chapter 6 put off a test for the last-table race, and chapter 20’s SQL has only ever been run by hand. Both get tests in this chapter, and one of the new tests turns up a bug that has been in the booking rules since chapter 4.

The chapter needs five new libraries, all for tests only. In build.sbt, add one line to the list for core, with a comma after the munit above it:

      "org.scalameta" %% "munit-scalacheck" % "1.3.1" % Test

and four to the list for service, again with a comma after its munit:

      "org.typelevel" %% "munit-cats-effect" % "2.2.1" % Test,
      "org.typelevel" %% "cats-effect-testkit" % "3.7.1" % Test,
      "com.dimafeng" %% "testcontainers-scala-munit" % "0.44.1" % Test,
      "com.dimafeng" %% "testcontainers-scala-postgresql" % "0.44.1" % Test

21.1. Tests that return IO

Our tests of an IO have ended in unsafeRunSync() since chapter 13, and needed an import to supply the threads. With munit-cats-effect a test hands its IO back and the suite runs it. This chapter’s test files for the service all go in service/src/test/scala. Replace api.test.scala there with this version:

package fullybooked

import cats.effect.IO
import cats.syntax.all.*
import sttp.model.StatusCode

class ApiTests extends munit.CatsEffectSuite:
  def answers(requests: List[NewBooking]): IO[List[Either[StatusCode, String]]] =
    for
      bookings <- Bookings.starting(regulars)
      results <- requests.traverse(request => create(bookings, request))
    yield results.map(result => result.left.map(refused => refused(0)).map(view => view.name))

  test("a good request is booked"):
    assertIO(answers(List(NewBooking("Ibrahim", 7, "20:00", None))), List(Right("Ibrahim")))

  test("a request that fails validation is a 400 listing every problem"):
    val bad = NewBooking("", 7, "8pm", Some("ask for Dev"))
    for
      bookings <- Bookings.starting(regulars)
      result <- create(bookings, bad)
    yield assertEquals(
      result.left.map(refused => (refused(0), refused(1).problems.size)),
      Left((StatusCode.BadRequest, 3))
    )

  test("the second party after the last table gets a 409"):
    val both = List(
      NewBooking("Ibrahim", 7, "20:00", None),
      NewBooking("Castellanos", 8, "20:00", None)
    )
    assertIO(answers(both), List(Right("Ibrahim"), Left(StatusCode.Conflict)))

  test("a name with no booking is a 404"):
    val program = Bookings.starting(regulars).flatMap(bookings => find(bookings, "Smith"))
    assertIO(program.map(_.left.map(refused => refused(0))), Left(StatusCode.NotFound))
$ sbt "service/test fullybooked.ApiTests"
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] elapsed time: 2 s

The suite extends munit.CatsEffectSuite, whose assertIO runs a program and compares its result with a value. A test can also be a for that ends in ordinary assertions, like the second one here. Either way the body’s type is IO, so a test that forgets to run something no longer passes by accident. The other suites can be converted by changing the class they extend, returning the IO from each test (usually through assertIO) and deleting the unsafeRunSync() calls with their import. I’d do each suite the next time I had a reason to open it.

21.2. Time under control

Chapter 15’s retrying waits between attempts. confirm allows three attempts and gives the gateway a fifth of a second to answer each time. The tests for retrying used pauses of a millisecond to stay quick, and nothing tested the timings in confirm at all. Create time.test.scala:

package fullybooked

import cats.effect.IO
import cats.effect.testkit.TestControl
import java.io.IOException
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import scala.concurrent.duration.*

class TimeTests extends munit.CatsEffectSuite:
  def timed[A](program: IO[A]): IO[FiniteDuration] =
    for
      start <- IO.monotonic
      _ <- program.attempt
      end <- IO.monotonic
    yield end - start

  test("an hour's sleep takes no real time"):
    assertIO(TestControl.executeEmbed(timed(IO.sleep(1.hour))), 1.hour)

  test("retrying pauses for 20, 40 and 80 milliseconds before it gives up"):
    val down = IO.raiseError[Unit](IOException("gateway unavailable"))
    val program = timed(retrying(down, attempts = 4, pause = 20.millis))
    assertIO(TestControl.executeEmbed(program), 140.millis)

  test("a gateway that never answers takes three timeouts and two pauses"):
    val logger: Logger[IO] = Slf4jLogger.getLogger[IO]
    val phone = Phone.parse("07700 900164")
    val booking = Booking(Party("Moreau", 3, phone), Table(3, 4), eight, Status.Confirmed)
    val program =
      for
        calls <- IO.ref(Map[Phone, Int]())
        gateway = FlakyGateway(calls, failsTwice = Set(), hangs = phone.toSet)
        elapsed <- timed(confirm(gateway, logger, booking))
        made <- calls.get
      yield (elapsed, made.values.sum)
    assertIO(TestControl.executeEmbed(program), (660.millis, 3))
$ sbt "service/test fullybooked.TimeTests"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

A test can now sleep for an hour and take no real time, as the first one does. TestControl.executeEmbed runs a program on a runtime where TestControl is in charge of the clock. When every fiber is asleep it moves the clock to the next moment anything is due to wake, so no real time passes. The helper timed measures how long a program took by reading that clock with IO.monotonic.

Because the test’s clock only moves when its fibers sleep, a test can assert exact times. Four attempts with a pause that starts at 20 milliseconds and doubles should wait for 20, 40 and 80. The test of retrying checks that the total is 140. The last test puts the Moreaus’ number behind a gateway that hangs. Three timeouts of 200 milliseconds and pauses of 20 and 40 come to 660, after three calls. The WARN line that test prints is confirm reporting that the gateway didn’t answer.

21.3. Properties

Our tests so far have named their inputs: a party of four at eight, a party of nine. A property test states something that should hold for any input and lets a library, here ScalaCheck, look for a counter-example. Create rules.test.scala in core/src/test/scala:

package fullybooked

import java.time.LocalTime
import org.scalacheck.Gen
import org.scalacheck.Prop.forAll

class RuleProperties extends munit.ScalaCheckSuite:
  case class Asked(size: Int, time: LocalTime)

  val request: Gen[Asked] =
    for
      size <- Gen.choose(1, 10)
      quarter <- Gen.choose(0, 95)
    yield Asked(size, LocalTime.MIDNIGHT.plusMinutes(quarter * 15L))

  def book(requests: List[Asked]): List[Booking] =
    requests.foldLeft(List[Booking]()): (bookings, asked) =>
      decide(bookings, Party("Guest", asked.size), asked.time) match
        case Right(booking) => bookings :+ booking
        case Left(_) => bookings

  def minutes(time: LocalTime): Int = time.getHour * 60 + time.getMinute

  def clash(a: Booking, b: Booking): Boolean =
    a.table == b.table &&
      minutes(a.time) < minutes(b.time) + b.party.stayMinutes &&
      minutes(b.time) < minutes(a.time) + a.party.stayMinutes

  property("no two parties are given the same table at the same time"):
    forAll(Gen.listOf(request)): requests =>
      val booked = book(requests)
      val clashes = for
        (a, i) <- booked.zipWithIndex
        b <- booked.drop(i + 1)
        if clash(a, b)
      yield (a.time, b.time, a.table.number)
      assertEquals(clashes, List())

A Gen describes how to make random values. The one called request makes an Asked, a party of one to ten at any quarter of an hour in the day. It’s built by combining two smaller generators in a for.

The property is checked against whole lists of requests, which Gen.listOf(request) generates and forAll passes to the block beneath it. The block puts a list through decide one request after another with book, then looks for two of the resulting bookings that share a table at overlapping times. zipWithIndex numbers the bookings so that each pair is looked at once. The overlap check, clash, works in minutes since midnight and shares no code with freeFor, so a mistake in freeFor is unlikely to be repeated in the test.

$ sbt "core/test fullybooked.RuleProperties"
Falsified after 26 passed tests.
> ARG_0: List("Asked(8,23:45)", "Asked(7,22:15)")
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0

ScalaCheck tries a hundred random lists, and this time the twenty-seventh failed. It then shrank that list, dropping requests for as long as the property still failed, which left two. ScalaCheck shrinks numbers as well, and in a plain tuple it could have taken the party size below one. A request is an Asked because ScalaCheck has no way to shrink a class of ours. Yours will probably be a different pair, because the lists are random. Mine is a party of eight at quarter to midnight, then a party of seven at quarter past ten, and book gives both of them table 6.

The party of eight is due to leave two hours later, at quarter to two. LocalTime has no days in it, so leaveBy is 01:45. Since that’s before 22:15, freeFor concludes that the table is free. A stay that runs past midnight hadn’t occurred to me, so I never wrote a test for one. Over HTTP the same bug gives a 500 for a POST at 22:45, because Postgres won’t build a range that ends before it starts. The restaurant stops taking orders at half past nine, but that rule is in whenOpen from chapter 7, which decide never calls.

To fix it, decide has to refuse a time when the kitchen is closed. If you did the first exercise in chapter 9, you have the error already. Otherwise add case KitchenClosed(time: LocalTime) to the end of BookingError in errors.scala. Compile, and -Werror points at the next thing to change: explain no longer covers every case. Add this to the end of it:

  case BookingError.KitchenClosed(time) => s"the kitchen is closed at $time"

Then, in decide in availability.scala, put a second check between the first and the else:

  if party.size > Booking.largestParty then Left(BookingError.PartyTooLarge(party.size))
  else if time.isBefore(opens) || time.isAfter(lastOrders) then
    Left(BookingError.KitchenClosed(time))

In routes.scala, create answers 409 for anything reserve refuses, but a closed kitchen is the client’s mistake and should be 400. A party too large for any table should be 400 too. Add this function above find, and change the last line of create to call it in place of StatusCode.Conflict. If you wrote statusFor in chapter 19’s exercises, give that the new case and keep it instead:

def statusOf(error: BookingError): StatusCode = error match
  case BookingError.NothingFree(_, _) | BookingError.NotAllowed(_, _) => StatusCode.Conflict
  case BookingError.PartyTooLarge(_) | BookingError.KitchenClosed(_) | BookingError.NameMissing |
      BookingError.BadPartySize(_) | BookingError.BadTime(_) | BookingError.BadPhone(_) =>
    StatusCode.BadRequest

With the bug fixed, add two more properties to the end of RuleProperties. The second one’s input is Gen.listOfN(11, Gen.numChar), a list of eleven random digits:

  property("nobody is seated at a table too small for them"):
    forAll(Gen.listOf(request)): requests =>
      assert(book(requests).forall(booking => booking.party.size <= booking.table.seats))

  property("a phone number survives being parsed and printed"):
    forAll(Gen.listOfN(11, Gen.numChar)): digits =>
      val typed = digits.mkString
      assertEquals(Phone.parse(typed).map(_.formatted.filterNot(_.isWhitespace)), Some(typed))
$ sbt "core/test fullybooked.RuleProperties"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

Add a test for the new answer to the end of ApiTests as well:

  test("a booking for half past eleven at night is a 400"):
    assertIO(
      answers(List(NewBooking("Night", 2, "23:30", None))),
      List(Left(StatusCode.BadRequest))
    )

Properties suit rules that can be said in a sentence and broken by inputs I wouldn’t think to type, which describes most of availability.scala. When one fails, the output includes a seed and a line to add to the suite. With that line in, ScalaCheck replays the same case while you fix the bug.

21.4. A real Postgres

Nothing tests the SQL yet, or the constraint, which only exists inside a database. Testcontainers starts a real Postgres in Docker for the length of a suite. Create postgres.test.scala:

package fullybooked

import cats.effect.IO
import cats.syntax.all.*
import com.dimafeng.testcontainers.PostgreSQLContainer
import com.dimafeng.testcontainers.munit.TestContainerForAll
import doobie.implicits.*
import org.testcontainers.utility.DockerImageName

class PostgresTests extends munit.CatsEffectSuite, TestContainerForAll:
  override val containerDef: PostgreSQLContainer.Def =
    PostgreSQLContainer.Def(dockerImageName = DockerImageName.parse("postgres:18"))

  def withBookings[A](use: PostgresBookings => IO[A]): IO[A] =
    withContainers: postgres =>
      val config = DatabaseConfig(postgres.jdbcUrl, postgres.username, postgres.password)
      migrate(config) >> transactor(config).use: xa =>
        val seed =
          sql"TRUNCATE bookings".update.run >> regulars.traverse_(booking => insert(booking).run)
        val busy = sql"SELECT 1 FROM pg_sleep(0.1)".query[Int].unique.transact(xa)
        seed.transact(xa) >> List.fill(8)(busy).parSequence_ >> use(PostgresBookings(xa))

  test("a booking comes back from the database as it went in"):
    withBookings: bookings =>
      for
        made <- bookings.reserve(Party("Ibrahim", 7, Phone.parse("07700 900141")), eight)
        stored <- bookings.all
      yield
        assert(made.isRight)
        assertEquals(stored.filter(_.party.name == "Ibrahim"), made.toList)

  test("twenty parties after the last table: one gets it"):
    withBookings: bookings =>
      val parties = List.tabulate(20)(n => Party(s"Party $n", 7))
      for
        answers <- parties.parTraverse(party => bookings.reserve(party, eight))
        stored <- bookings.all
      yield
        assertEquals(answers.count(_.isRight), 1)
        assertEquals(stored.count(_.table.number == 6), 1)
$ sbt "service/test fullybooked.PostgresTests"
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] elapsed time: 2 s

Extending TestContainerForAll as well as CatsEffectSuite starts a container before the suite’s first test and removes it after the last. The container comes from the image that containerDef names. Docker gives the container a port chosen at random on your machine, so it can’t collide with the database from chapter 20. withContainers passes the running container to our code, with its address and password. Our fixture, withBookings, runs the migrations against the container, opens a pool, empties the table and seats the regulars, so each test starts from the same evening.

A new pool opens its connections in the background. The fixture waits for them before it hands over the PostgresBookings, by keeping all eight busy for a tenth of a second. Each runs a query that returns the one row unique expects. Before I added that step the second test passed with the constraint taken away, which means its twenty reservations weren’t overlapping.

One test checks that a booking read back from the table equals the one reserve returned, using made.toList to turn the Either into a list of one booking. The other is the last-table race. Twenty parties go for table 6 at once through a real connection pool, and exactly one of them gets it. I checked that this test fails when it should. Without ON CONFLICT DO NOTHING on the insert it fails with the constraint’s error. With the second migration taken away, eight parties are told the table is theirs. If you try the same, run the tests with sbt service/testFull, because test doesn’t notice that a migration file has gone and says there are no tests to run.

The whole suite takes two or three seconds on my machine, about half of it Postgres starting, with twenty-odd lines of Testcontainers’ own logging on the way. It does need Docker, on the build server as well as on yours.

21.5. The service without a network

The tests in ApiTests call create directly, so nothing yet checks that the JSON has the field names a client expects. Nor does anything check that a Refusal turns into the status code it carries. We can check both without opening a port, because an http4s application is a function from a request to an IO of a response.

At the moment Server builds its application inside serve, where a test can’t reach it. Replace server.scala with a version that builds it in a function of its own, httpApp:

package fullybooked

import cats.effect.{IO, IOApp}
import com.comcast.ip4s.{ipv4, Port}
import org.http4s.HttpApp
import org.http4s.ember.server.EmberServerBuilder
import org.http4s.server.middleware
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import sttp.tapir.server.http4s.Http4sServerInterpreter
import sttp.tapir.swagger.bundle.SwaggerInterpreter

def httpApp(title: String, bookings: Bookings): HttpApp[IO] =
  val routes = bookingRoutes(bookings)
  val docs = SwaggerInterpreter().fromServerEndpoints[IO](routes, title, "0.1.0")
  Http4sServerInterpreter[IO]().toRoutes(routes ++ docs).orNotFound

object Server extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def run: IO[Unit] =
    for
      config <- loadConfig
      applied <- migrate(config.database)
      _ <- logger.info(s"$applied migrations applied")
      port <- portFrom(config)
      _ <- transactor(config.database).use(xa => serve(config, port, PostgresBookings(xa)))
    yield ()

  def portFrom(config: AppConfig): IO[Port] =
    IO.fromOption(Port.fromInt(config.port))(
      IllegalArgumentException(s"${config.port} isn't a port")
    )

  def serve(config: AppConfig, port: Port, bookings: Bookings): IO[Unit] =
    EmberServerBuilder
      .default[IO]
      .withHost(ipv4"0.0.0.0")
      .withPort(port)
      .withHttpApp(
        middleware.Logger.httpApp(logHeaders = false, logBody = false)(
          httpApp(config.restaurant, bookings)
        )
      )
      .build
      .useForever

Then create http.test.scala:

package fullybooked

import cats.effect.IO
import org.http4s.{MediaType, Method, Request}
import org.http4s.headers.`Content-Type`
import org.http4s.implicits.*

class HttpTests extends munit.CatsEffectSuite:
  def post(body: String): Request[IO] =
    Request[IO](Method.POST, uri"/bookings")
      .withEntity(body)
      .withContentType(`Content-Type`(MediaType.application.json))

  test("a booking made over HTTP can be read back over HTTP"):
    for
      bookings <- Bookings.starting(List())
      app = httpApp("Fully Booked", bookings)
      created <- app.run(post("""{"name":"Hughes","size":4,"time":"18:30"}"""))
      listed <- app.run(Request[IO](Method.GET, uri"/bookings"))
      body <- listed.as[String]
    yield
      assertEquals(created.status.code, 201)
      assertEquals(
        body,
        """[{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}]"""
      )

  test("the kitchen's hours reach the client as a 400"):
    for
      bookings <- Bookings.starting(List())
      response <- httpApp("Fully Booked", bookings).run(
        post("""{"name":"Night","size":2,"time":"23:30"}""")
      )
      body <- response.as[String]
    yield
      assertEquals(response.status.code, 400)
      assertEquals(body, """{"problems":["the kitchen is closed at 23:30"]}""")

  test("a path nobody serves is a 404"):
    for
      bookings <- Bookings.starting(List())
      response <- httpApp("Fully Booked", bookings).run(Request[IO](Method.GET, uri"/menu"))
    yield assertEquals(response.status.code, 404)
$ sbt "service/test fullybooked.HttpTests"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

Calling app.run(request) sends a request through Tapir’s decoding, our logic and the encoding of the answer without touching a socket. With the in-memory Bookings behind the application, these tests need no database either. post builds the request and gives it a body with withEntity. The header’s name is in backticks because that’s how Scala writes an identifier with a hyphen in it. uri"/bookings" is another literal that’s checked at compile time. The tests read each response’s body with as[String]. They compare the status as a number, because the name Status would mean our enum here and http4s has a type called Status too.

I don’t write the same number of each kind of test. Most of mine are plain tests of functions, because that’s where the rules are. A handful of properties go with the rules where an input I hadn’t thought of could break something. A few tests go through HTTP to pin down the JSON and the status codes, and a few go to a real database for the SQL and for whatever only the database enforces. Run them all:

$ sbt testFull
[info] Passed: Total 34, Failed 0, Errors 0, Passed 34
[info] Passed: Total 38, Failed 0, Errors 0, Passed 38
[info] Passed: Total 6, Failed 0, Errors 0, Passed 6
[success] elapsed time: 2 s

21.6. Exercises

Solutions are in the book’s downloadable code, under ch21/exercises.

  1. Rewrite the first two tests of ConfirmationTests, the ones about retrying, in a CatsEffectSuite, with pauses of a minute under TestControl. interceptMessageIO[IOException]("...")(program) passes when the program fails with that exception and message.

  2. Write a property saying that an empty restaurant, at any quarter of an hour between opens and lastOrders, gives a party of one to eight the smallest table that fits.

  3. Insert a row with a status of 'Vanished' straight into the table with sql, and test that all on a PostgresBookings then fails with the message from toBooking. withBookings keeps the transactor to itself, so this test needs withContainers and a transactor of its own.

  4. Test GET /bookings/Smith through httpApp, checking the status and the body.

22. Shipping it

Starting the service with sbt "service/runMain fullybooked.Server" needs sbt, the source code and a compiler on the machine. It also needs a database password that’s sitting in a file in the repository. In this chapter we package the service so that a machine with only Java 25 can run it, with the password taken out of the file. We then add endpoints that tell whoever operates the service how it’s doing, and finish with the service in a container beside its database.

22.1. A package

An sbt plugin called sbt-native-packager does the packaging. Add it to project/plugins.sbt:

addSbtPlugin("com.github.sbt" % "sbt-native-packager" % "1.13.0")

In build.sbt, switch it on for service and add two settings at the top of that subproject’s list:

lazy val service = project
  .dependsOn(core)
  .enablePlugins(JavaAppPackaging)
  .settings(
    Compile / mainClass := Some("fullybooked.Server"),
    bashScriptExtraDefines += """addJava "--sun-misc-unsafe-memory-access=allow"""",

The service subproject has eleven entry points left over from earlier chapters, so mainClass tells the plugin which one to start. The other setting adds a line to the start script that the plugin writes, to pass the JVM option we’ve been using since chapter 6. It’s in triple quotes because it has quotes of its own. Add one more line near the top of the file, under version:

Compile / doc / sources := Seq()

Without it, packaging also builds the API documentation for our code, which takes a little longer and prints a warning about a comment in somebody else’s library. Run sbt service/stage to build the package into a directory:

$ ls target/out/jvm/scala-3.9.0/service/universal/stage
bin
lib

The bin directory has a start script for each entry point, with a .bat version for Windows. The one called bin/service starts the server on a JVM with every jar in lib on its classpath. There are eighty of those jars: our code and every library it depends on.

22.2. Secrets from the environment

The database block that we added to application.conf in chapter 20 has a password in it. Give user an override and replace the password, so that the block reads:

database {
  url = "jdbc:postgresql://localhost:5432/fullybooked"
  url = ${?DATABASE_URL}
  user = "postgres"
  user = ${?DATABASE_USER}
  password = ${DATABASE_PASSWORD}
}

With no default and no question mark on the password line, the service can’t start unless DATABASE_PASSWORD is set. Run sbt service/stage again so that the package has the new file, then start the packaged service:

$ target/out/jvm/scala-3.9.0/service/universal/stage/bin/service
pureconfig.error.ConfigReaderException: Cannot convert configuration to a fullybooked.AppConfig. Failures are:
  - (application.conf @ .../application.conf: 11) Unable to parse the configuration: Could not resolve substitution to a value: ${DATABASE_PASSWORD}.

Those two lines come with a stack trace under them. Set the variable in your terminal with export DATABASE_PASSWORD=table-for-two. From now on every program that loads the configuration needs it, including LastTableDb and the server when they’re run through sbt. The tests don’t load the configuration, so they pass without the variable. In production, a platform such as Kubernetes runs the service and sets the variable from its own store of secrets.

22.3. Health and readiness

The platform that runs a service asks whether its process is alive, so that it can restart a dead one. It also asks whether the service can do its job right now, which for ours means being able to reach the database. When there are several copies of a service, requests go only to those that can. Add a method to the Bookings trait in concurrency.scala:

  def ready: IO[Boolean]

Add this to InMemoryBookings, below all, since that class is always ready:

  def ready: IO[Boolean] = IO.pure(true)

For PostgresBookings the answer comes from the database. Add import scala.concurrent.duration.* to the imports of postgres.scala, and add this method to the class, below the method called all:

  def ready: IO[Boolean] =
    sql"SELECT 1".query[Int].unique.transact(xa).timeout(2.seconds).attempt.map(_.isRight)

The database gets two seconds to answer SELECT 1, a query that reads no table. With attempt, a failure or a timeout becomes a Left, and ready answers false.

The two seconds in ready don’t cover the wait for a connection. Getting one from the pool is a blocking call, and chapter 15 showed that a timeout can’t interrupt one of those. With the database gone, the pool goes on trying to open a connection for thirty seconds by default, and /ready would take that long to answer. In transactor in database.scala, add a line under the one that sets the size of the pool:

  pool.setConnectionTimeout(2000)

The figure is in milliseconds. It covers every wait for a connection, so a booking made while the database is down no longer keeps the client waiting for thirty seconds.

Two endpoints go at the end of api.scala:

val healthEndpoint = endpoint.get.in("health").out(stringBody)

val readyEndpoint =
  endpoint.get
    .in("ready")
    .out(stringBody)
    .errorOut(statusCode(StatusCode.ServiceUnavailable))
    .errorOut(stringBody)

and their logic at the end of routes.scala:

def opsRoutes(bookings: Bookings): List[ServerEndpoint[Fs2Streams[IO], IO]] =
  List(
    healthEndpoint.serverLogicSuccess(_ => IO.pure("ok")),
    readyEndpoint.serverLogic: _ =>
      bookings.ready.map(up => if up then Right("ready") else Left("the database isn't answering"))
  )

Both answer in plain text, which is what stringBody describes. /health doesn’t touch the database, because a slow database is no reason to restart the service. /ready answers 503 when the database can’t be reached.

22.4. Metrics, a time limit and a clean stop

Add two more libraries to the service list in build.sbt, after the line for Flyway:

      "org.http4s" %% "http4s-prometheus-metrics" % "0.25.0",
      "net.logstash.logback" % "logstash-logback-encoder" % "9.0",

Replace server.scala once more:

package fullybooked

import cats.effect.{IO, IOApp}
import cats.syntax.all.*
import com.comcast.ip4s.{ipv4, Port}
import org.http4s.{HttpApp, HttpRoutes}
import org.http4s.ember.server.EmberServerBuilder
import org.http4s.metrics.prometheus.{Prometheus, PrometheusExportService}
import org.http4s.server.middleware
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import scala.concurrent.duration.*
import sttp.tapir.server.http4s.Http4sServerInterpreter
import sttp.tapir.swagger.bundle.SwaggerInterpreter

def httpRoutes(title: String, bookings: Bookings): HttpRoutes[IO] =
  val routes = bookingRoutes(bookings)
  val docs = SwaggerInterpreter().fromServerEndpoints[IO](routes, title, "0.1.0")
  Http4sServerInterpreter[IO]().toRoutes(routes ++ docs ++ opsRoutes(bookings))

def httpApp(title: String, bookings: Bookings): HttpApp[IO] =
  httpRoutes(title, bookings).orNotFound

object Server extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def run: IO[Unit] =
    val service =
      for
        config <- loadConfig
        applied <- migrate(config.database)
        _ <- logger.info(s"$applied migrations applied")
        port <- portFrom(config)
        _ <- transactor(config.database).use(xa => serve(config, port, PostgresBookings(xa)))
      yield ()
    service.guarantee(logger.info("Fully Booked has stopped"))

  def portFrom(config: AppConfig): IO[Port] =
    IO.fromOption(Port.fromInt(config.port))(
      IllegalArgumentException(s"${config.port} isn't a port")
    )

  def serve(config: AppConfig, port: Port, bookings: Bookings): IO[Unit] =
    val server =
      for
        exporter <- PrometheusExportService.build[IO]
        ops <- Prometheus.metricsOps[IO](exporter.collectorRegistry, "fully_booked")
        measured = middleware.Metrics[IO](ops)(httpRoutes(config.restaurant, bookings))
        app = (exporter.routes <+> measured).orNotFound
        limited = middleware.Timeout.httpApp[IO](10.seconds)(app)
        server <- EmberServerBuilder
          .default[IO]
          .withHost(ipv4"0.0.0.0")
          .withPort(port)
          .withHttpApp(middleware.Logger.httpApp(logHeaders = false, logBody = false)(limited))
          .withShutdownTimeout(5.seconds)
          .build
      yield server
    server.useForever

Most of httpRoutes is the old httpApp. The two new endpoints join the list after the documentation has been built, and I leave them out of it because the people who call /health and /ready are operating the service. The new function returns the routes before orNotFound has turned them into an application, since routes can still be joined to other routes and an application can’t. The tests still call httpApp, which now only adds orNotFound.

The for in serve runs over three resources: the exporter, the ops value that records the numbers, and the server. A for over Resources makes a single Resource, which releases the server first and the exporter last, the reverse of the order they were acquired in. The first two lines are for Prometheus, a widely used monitoring system that collects a service’s numbers by fetching a page of them at regular intervals. The Metrics middleware counts every request to our routes and times every response. The exporter serves those numbers at /metrics from routes of its own, which are joined to ours with <+>. That operator tries the routes on its left first. I’ve put the exporter’s there to keep Prometheus’s own fetches out of the counts.

The Timeout middleware goes round the whole application, metrics included. A request that hasn’t been answered in ten seconds gets a 503, and the fiber working on it is cancelled. When a fiber is cancelled doobie cancels its query, so a stuck query can’t hold a connection for ever.

When the process is told to stop, IOApp cancels run, and that releases every Resource, the server first. The server gives connections that are still open five seconds to finish, the figure passed to withShutdownTimeout. That’s half of the ten seconds Docker waits before it kills a container that hasn’t stopped. Then the pool closes and the guarantee in run logs a line.

Add a test for the new endpoints to the end of HttpTests:

  test("the service says it's alive and ready"):
    for
      bookings <- Bookings.starting(List())
      app = httpApp("Fully Booked", bookings)
      health <- app.run(Request[IO](Method.GET, uri"/health"))
      ready <- app.run(Request[IO](Method.GET, uri"/ready"))
      answer <- ready.as[String]
    yield
      assertEquals(health.status.code, 200)
      assertEquals((ready.status.code, answer), (200, "ready"))
$ sbt "service/test fullybooked.HttpTests"
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] elapsed time: 2 s

22.5. Logs for machines

Our log lines are laid out for a person reading a terminal. In production they usually go to a program first, and it wants one JSON object per line. The second of the two libraries we added to build.sbt in the last section, the Logstash encoder for Logback, writes them in that form. Create logback-json.xml beside logback.xml:

<configuration>
  <statusListener class="ch.qos.logback.core.status.NopStatusListener"/>
  <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>
  <logger name="org.flywaydb" level="WARN"/>
  <logger name="com.zaxxer.hikari" level="WARN"/>
  <root level="INFO">
    <appender-ref ref="console"/>
  </root>
</configuration>

Logback goes on using logback.xml unless the system property logback.configurationFile names another file. We’ll set that property only in the container.

22.6. An image

Run sbt service/stage again to pick up the changes. Then create a file called Dockerfile in the project’s top directory:

FROM eclipse-temurin:25-jre

RUN useradd --system --no-create-home fullybooked
WORKDIR /app
COPY --chmod=755 target/out/jvm/scala-3.9.0/service/universal/stage/ /app/
USER fullybooked

ENV JAVA_OPTS="-Dlogback.configurationFile=logback-json.xml"
EXPOSE 8080
ENTRYPOINT ["/app/bin/service"]

The image starts from one that has a Java 25 runtime and nothing to compile with. The RUN line makes a user for the service so that the process doesn’t run as root, and COPY brings the staged directory in. --chmod=755 is there because the packager makes its scripts executable for their owner alone, and in the image the owner is root. The bin/service script reads JAVA_OPTS, which here holds the system property that names the JSON logging file. Create a .dockerignore file beside the Dockerfile to keep everything except the staged directory out of the build:

**
!target/out/jvm/scala-3.9.0/service/universal/stage/**

sbt-native-packager can write a Dockerfile and build the image itself. I write the file by hand, because it’s a few lines and the people who run the service can read it.

22.7. The service and its database together

Docker Compose starts several containers from one description. Create compose.yaml in the same directory:

services:
  db:
    image: postgres:18
    environment:
      POSTGRES_DB: fullybooked
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "postgres", "-d", "fullybooked"]
      interval: 2s
      retries: 15

  service:
    build: .
    environment:
      DATABASE_URL: jdbc:postgresql://db:5432/fullybooked
      DATABASE_PASSWORD: ${DATABASE_PASSWORD}
    ports:
      - "8080:8080"
    depends_on:
      db:
        condition: service_healthy

Both passwords come from the DATABASE_PASSWORD in your terminal, so export the variable again if you’ve opened a new one. Inside Compose’s network the database’s host name is db, the name in the URL. Because of depends_on, the service isn’t started until Postgres answers pg_isready. Port 8080 has to be free, so stop any server that’s still running from an earlier chapter. Then build and start everything:

$ docker compose up -d --build

Give the service a couple of seconds to start, then ask it how it is:

$ curl localhost:8080/health
ok
$ curl localhost:8080/ready
ready

Book a table with one of chapter 19’s curl commands, then look at what the service has counted:

$ curl -s localhost:8080/metrics | grep ^fully_booked_request_count_total
fully_booked_request_count_total{classifier="",method="get",status="2xx",} 2.0
fully_booked_request_count_total{classifier="",method="post",status="2xx",} 1.0

The 2 beside get counts the requests just made to /health and /ready. The logs are JSON now:

$ docker compose logs --no-log-prefix service
...
{"@timestamp":"2026-11-14T19:30:00.000Z","@version":"1","message":"HTTP/1.1 POST /bookings","logger_name":"org.http4s.server.middleware.Logger","thread_name":"io-compute-1","level":"INFO","level_value":20000}
{"@timestamp":"2026-11-14T19:30:00.000Z","@version":"1","message":"HTTP/1.1 201 Created","logger_name":"org.http4s.server.middleware.Logger","thread_name":"io-compute-1","level":"INFO","level_value":20000}
...

Stop the database underneath the service, wait a few seconds and ask again:

$ docker compose stop db
$ curl -w '\n%{http_code}\n' localhost:8080/ready
the database isn't answering
503

A request to /health still gets ok, because that endpoint never asks the database. Stop the service and read the end of its log:

$ docker compose stop service
$ docker compose logs --no-log-prefix service
...
{"@timestamp":"2026-11-14T19:30:00.000Z","@version":"1","message":"Fully Booked has stopped","logger_name":"fullybooked.Server","thread_name":"io-compute-1","level":"INFO","level_value":20000}

Run docker compose down to remove the containers.

22.8. Exercises

Solutions to the first two are in the book’s downloadable code, under ch22/exercises.

  1. The pool has eight connections because database.scala says so. Add a pool-size setting to the database block, with a default of 8 and an environment variable DATABASE_POOL_SIZE to override it, and use it in transactor. The fixture in postgres.test.scala builds a DatabaseConfig as well and will need the new field.

  2. Write a test that gives httpApp a Bookings whose ready is false, and check that /ready answers 503 while /health still answers 200. new Bookings: followed by the three methods makes one on the spot.

  3. With everything running under Compose, stop the database and try to book a table. What does the client get, and how long does it wait? Which of chapter 15’s tools would improve on that, and where would it go?

23. Play: the Scala you’ll inherit

Part IV has built a service the way I’d build one now. A Scala web application that has been running for ten years was probably built on the Play framework. In outline Play is close to Spring MVC: there’s a file of routes, a controller class with a method for each route and a dependency injection container that builds the controllers. Underneath it’s asynchronous, built on the Future type from Scala’s standard library.

In this chapter we rebuild two of chapter 19’s endpoints in Play 3, on top of the booking rules in core, so that you can find your way around a Play application when you’re handed one.

23.1. A Future has already started

A Play controller answers each request with a Future. A Future needs nothing from Play, so try one in a script first. Save this as futures.scala in a directory of its own and run it:

//> using scala 3.9.0
//> using jvm 25

import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.*

@main def futures(): Unit =
  val table = Future:
    println("looking for a table")
    6
  Thread.sleep(100)
  println("the Future has been made, and nothing has asked for its result")
  val twice =
    for
      first <- table
      second <- table
    yield first + second
  println(Await.result(twice, 1.second))
$ scala run futures.scala
looking for a table
the Future has been made, and nothing has asked for its result
12

Like IO.delay, Future takes a block of code, but the block doesn’t wait for anything to run it. Here the block ran as soon as the Future was made, on another thread. Its message is the first line of output, ahead of the one printed after the sleep. That thread came from an ExecutionContext, Scala’s name for a thread pool that futures run on. The import of global supplies a default one as a given.

The for then uses table twice, yet the message was printed once. A Future runs once and keeps its result, so flatMap on a finished one hands over the value it already has. An IO used twice in a for would have printed the message twice.

The retrying we wrote in chapter 15 took a program and ran it again if it failed, up to a number of attempts. Handed a Future, it could only read the same result again, so code that retries futures takes a function that makes a new Future on each call. A Future can’t be cancelled either, and a timeout on one stops the waiting while the work carries on. Exercise 4 has you swap a val holding a Future for a def and see how many times its block then runs.

The last line’s Await.result blocks the current thread until the Future has finished, so that the script doesn’t end first. A Play controller never needs to wait, because it hands its Future to the framework and returns.

23.2. The rules as a library

The Play application is a separate project that uses core as a library. It can’t be a fourth subproject, because the stable release of Play, 3.0, has an sbt plugin for sbt 1 and not yet for sbt 2. In the Fully Booked directory, publish core to a repository on your own machine. These are two of the lines sbt prints, with ~ standing for your home directory:

$ sbt core/publishLocal
[info] 	published core_3 to ~/.ivy2/local/com.fullybooked/core_3/0.1.0/jars/core_3.jar
[success] elapsed time: 2 s

The organisation and version at the top of build.sbt have been there since chapter 18. With the subproject’s name they make up what another build asks for: com.fullybooked, core and 0.1.0. The publishLocal task puts the library under ~/.ivy2/local, where only builds on this machine will find it. A team would normally publish to a repository that all of its builds can reach.

23.3. A Play project

Make a new directory called fully-booked-play beside the other one. Its build starts with project/build.properties, which asks for sbt 1:

sbt.version=1.13.1

Play’s plugin goes in project/plugins.sbt:

addSbtPlugin("org.playframework" % "sbt-plugin" % "3.0.12")

The build.sbt goes in the new directory itself:

name := "fully-booked-play"
scalaVersion := "3.9.0"

lazy val root = (project in file(".")).enablePlugins(PlayScala)

libraryDependencies ++= Seq(
  guice,
  "com.fullybooked" %% "core" % "0.1.0",
  "org.scalatestplus.play" %% "scalatestplus-play" % "7.0.2" % Test
)

Test / javaOptions += "--sun-misc-unsafe-memory-access=allow"

The sbt command is the launcher you already have, which reads build.properties and starts sbt 1.13.1 for this directory. In build.sbt, project in file(".") makes the directory itself the project, the usual way to write a build with no subprojects. Turning on PlayScala brings in Play’s libraries, its directory layout and a run task of its own. The plugin also defines guice, its name for the dependency injection library Play uses. The line below it asks for the core we’ve just published. The last setting passes the JVM option from chapter 6 to the tests. From here on sbt prints Play’s banner before every command, which the output shown in this chapter leaves out.

Play has a directory layout of its own in place of Maven’s. Code goes under app, in packages named for what the classes are, such as controllers, models and services. Configuration and the routes go in conf, and tests in test. Create an empty file at conf/application.conf. Ours has nothing to set, but without the file Play answers every request with a configuration error. An application that’s been in production for years will probably have a long application.conf, in the format of the one we wrote for the http4s service.

23.4. Two endpoints

The JSON types go in app/models/views.scala:

package models

import fullybooked.{describe, Booking}
import play.api.libs.json.{Json, OFormat}

case class NewBooking(name: String, size: Int, time: String, phone: Option[String])

case class BookingView(name: String, size: Int, table: Int, time: String, status: String)

object NewBooking:
  given OFormat[NewBooking] = Json.format[NewBooking]

object BookingView:
  given OFormat[BookingView] = Json.format[BookingView]

  def of(booking: Booking): BookingView =
    BookingView(
      booking.party.name,
      booking.party.size,
      booking.table.number,
      booking.time.toString,
      describe(booking.status)
    )

These are chapter 19’s NewBooking and BookingView with Play’s own JSON library, play-json, behind them in place of circe. In chapter 19 a derives clause had circe write the code that turns a case class into JSON and back. Here Json.format writes it, also at compile time, as an OFormat. Each OFormat is a given in its class’s companion object, which is one of the places the compiler looks for givens.

The bookings are held by a class in app/services/BookingBook.scala:

package services

import fullybooked.{decide, Booking, BookingError, Party}
import java.time.LocalTime
import javax.inject.Singleton
import scala.concurrent.Future

@Singleton
class BookingBook:
  private var bookings = List[Booking]()

  def all: Future[List[Booking]] =
    Future.successful(synchronized(bookings))

  def reserve(party: Party, time: LocalTime): Future[Either[BookingError, Booking]] =
    Future.successful:
      synchronized:
        decide(bookings, party, time).map: booking =>
          bookings = bookings :+ booking
          booking

As in chapter 17’s DirectBookings, the list is a var that’s only read or changed inside synchronized. Each answer is wrapped in Future.successful, which makes a Future that has already finished. In an application with a database the futures would come from the database library: Slick, which Play has a module for, returns a Future from every query it runs. I’ll come back to @Singleton once the application runs.

The controller is app/controllers/BookingController.scala:

package controllers

import fullybooked.{explain, validate, BookingError, BookingRequest}
import javax.inject.{Inject, Singleton}
import models.{BookingView, NewBooking}
import play.api.libs.json.Json
import play.api.mvc.{AbstractController, ControllerComponents, Result}
import scala.concurrent.{ExecutionContext, Future}
import services.BookingBook

@Singleton
class BookingController @Inject() (book: BookingBook, components: ControllerComponents)(using
    ExecutionContext
) extends AbstractController(components):

  def list() = Action.async:
    book.all.map(bookings => Ok(Json.toJson(bookings.map(BookingView.of))))

  def create() = Action.async(parse.json[NewBooking]): request =>
    val asked = request.body
    val asText =
      BookingRequest(asked.name, asked.size.toString, asked.time, asked.phone.getOrElse(""))
    validate(asText) match
      case Left(errors) =>
        Future.successful(BadRequest(Json.obj("problems" -> errors.toList.map(explain))))
      case Right((party, time)) =>
        book
          .reserve(party, time)
          .map:
            case Right(booking) => Created(Json.toJson(BookingView.of(booking)))
            case Left(error) => refused(error)

  private def refused(error: BookingError): Result =
    val problems = Json.obj("problems" -> List(explain(error)))
    error match
      case BookingError.NothingFree(_, _) | BookingError.NotAllowed(_, _) => Conflict(problems)
      case BookingError.PartyTooLarge(_) | BookingError.KitchenClosed(_) |
          BookingError.NameMissing | BookingError.BadPartySize(_) | BookingError.BadTime(_) |
          BookingError.BadPhone(_) =>
        BadRequest(problems)

Each public method of the controller returns an action, a function from a request to a Future of a result, and Action.async builds one from a block. The controller gets Action from AbstractController, along with parse and the functions that make results: Ok, Created, BadRequest and Conflict, named for their status codes. A result’s body here is JSON, made either by Json.toJson with the formats from views.scala or by Json.obj from pairs of names and values. I’ve left the return types off the two actions, as Play code usually does.

The create method goes through the steps of chapter 19’s create. It turns the NewBooking into a BookingRequest and calls validate from core. If that fails it answers 400 with the output of explain. When reserve turns a booking down, the refused method picks 409 or 400, as statusOf does in the http4s service. With parse.json[NewBooking], Play decodes the body before our block runs. The map near the end is on the Future that reserve returns, so its function runs on a thread from the ExecutionContext in the class’s using clause. That parameter has no name because nothing in the class refers to it, and map finds it as a given.

The routes go in conf/routes, one line for each request and the method that answers it:

GET     /bookings            controllers.BookingController.list()
POST    /bookings            controllers.BookingController.create()

23.5. Running it

Start Play with sbt run. Play listens on port 9000 in what it calls development mode, where nothing is compiled until the first request arrives:

$ curl localhost:9000/bookings
[]
$ curl -d '{"name":"Hughes","size":4,"time":"18:30"}' -H 'Content-Type: application/json' localhost:9000/bookings
{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}
$ curl -d '{"name":"","size":4,"time":"7pm"}' -H 'Content-Type: application/json' -w '\n%{http_code}\n' localhost:9000/bookings
{"problems":["we need a name for the booking","'7pm' isn't a time, try 19:30"]}
400

Leave it running and change something in the controller. On the next request Play recompiles what changed and starts the application again, which also means a new, empty BookingBook. Then break the controller by misspelling book.all and open http://localhost:9000/bookings in a browser. The compile error is the response, on a page that shows the offending line. Put the spelling right afterwards and leave the server running. When it’s time to stop it, press Enter in sbt’s terminal.

A body that isn’t a NewBooking never reaches create, because Play answers it first, as Tapir did in chapter 19. This is the start of Play’s answer:

$ curl -s -d '{"name":"Tanaka","size":"eight","time":"20:00"}' -H 'Content-Type: application/json' localhost:9000/bookings | head -n 4
<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Bad Request</title>

The status is 400 and the body is a page of HTML, since Play’s default error handler is written for browsers. Play has a JSON one as well, chosen with play.http.errorHandler = play.api.http.JsonHttpErrorHandler in application.conf. For an API I’d set that on the first day.

Stop the server, change list() to listAll() in conf/routes and compile:

$ sbt compile
[error] conf/routes:1:1: value listAll is not a member of controllers.BookingController
[error] GET     /bookings            controllers.BookingController.listAll()
[error] one error found
[error] (Compile / compileIncremental) Compilation failed

Play turns the routes into Scala before the compiler runs, so a route to a method that doesn’t exist fails the build with the routes file’s own line number in the error. Put list() back.

23.6. Who calls the constructor

Guice makes the BookingController and hands it a BookingBook when the application starts. The @Inject() annotation marks the constructor that Guice should call. Guice finds a value for each parameter by its type: it builds the BookingBook itself, and Play provides the ControllerComponents and the ExecutionContext for the using clause. With @Singleton, there’s one BookingBook for the whole application, where Guice’s default is a new one for every place that wants it.

In chapter 19 bookingRoutes(bookings) got what it depended on as an argument, and the compiler checked that it was there. Guice does its wiring at run time, so a constructor parameter that it can’t supply still compiles and the application fails as it starts. It’s the reason I prefer plain arguments, though in a Play codebase injection is the convention and I follow it.

In Scala 2 the controller’s last parameter list reads (implicit ec: ExecutionContext), with implicit where Scala 3 has using.

23.7. Testing a controller

The last dependency in build.sbt, scalatestplus-play, has Play’s test helpers for ScalaTest, the framework we saw a sample of in chapter 6. Create test/controllers/BookingControllerSpec.scala:

package controllers

import org.scalatestplus.play.PlaySpec
import org.scalatestplus.play.guice.GuiceOneAppPerTest
import play.api.libs.json.Json
import play.api.test.FakeRequest
import play.api.test.Helpers.*

class BookingControllerSpec extends PlaySpec with GuiceOneAppPerTest:
  def booking(name: String, size: Int, time: String) =
    FakeRequest(POST, "/bookings")
      .withJsonBody(Json.obj("name" -> name, "size" -> size, "time" -> time))

  "POST /bookings".should:
    "book a table that can then be listed".in:
      val created = route(app, booking("Hughes", 4, "18:30")).get
      status(created).mustBe(CREATED)

      val listed = route(app, FakeRequest(GET, "/bookings")).get
      (contentAsJson(listed) \\ "name").map(_.as[String]).mustBe(Seq("Hughes"))

    "refuse the second party after the last table".in:
      status(route(app, booking("Mbeki", 8, "20:00")).get).mustBe(CREATED)
      status(route(app, booking("Tanaka", 8, "20:00")).get).mustBe(CONFLICT)

    "say what was wrong with a bad request".in:
      val refused = route(app, booking("", 4, "7pm")).get
      status(refused).mustBe(BAD_REQUEST)
      (contentAsJson(refused) \ "problems").as[Seq[String]].size.mustBe(2)
$ sbt test
[info] BookingControllerSpec:
[info] POST /bookings
[info] - should book a table that can then be listed
[info] - should refuse the second party after the last table
[info] - should say what was wrong with a bad request
[info] Tests: succeeded 3, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

I’ve left out the log lines that Play prints between the test names. The suite extends PlaySpec, a ScalaTest suite with Play’s helpers added. Its other parent, GuiceOneAppPerTest, has Guice build a whole application before each test, so every test starts with an empty BookingBook. A test’s route(app, request) sends the request through the routes file to the controller without a network. Since a request might match no route, the result is an Option of a Future. Calling .get on it gives the Future, or throws if there was no route. Play’s status and contentAsJson wait for that Future. In the JSON, \ picks out a field and \\ collects every field of that name at any depth. The as method reads a piece of JSON as a Scala type.

I’ve written .should:, .in: and .mustBe with dots, to match the rest of the book. ScalaTest’s own documentation has them as bare words, as in status(created) mustBe CREATED, with braces around the blocks. An inherited codebase will probably use that style.

23.8. Working in a codebase you didn’t choose

A Play application that you’re given will probably be written in Scala 2, the subject of appendix A, with braces and implicit throughout. The application may also be on Play 2. Play 2.9 and 3.0 are the same framework, the first running on Akka and the second on Pekko, the open-source fork that was made after Akka’s licence changed in 2022. In the code, imports start with akka on Play 2 and with org.apache.pekko on Play 3.

You probably won’t be able to switch on chapter 18’s compiler options all at once. With -Wunused:all this application of two routes gets five warnings, every one of them in code that Play generated from the routes file. I’d still turn the warnings on and work through the ones in my own files before thinking about -Werror.

When one of these applications is slow, I look for blocking first. Play runs controllers on a small pool of threads that it expects never to wait. A JDBC call made inside a Future on that pool holds one of its threads for the length of the query. Cats Effect has IO.blocking for that problem, and Play’s answer is a second ExecutionContext kept for blocking work. I’d read the page on thread pools in its documentation before changing any database code.

With an inherited application I’d begin by moving the rules out of the controllers into plain functions with tests around them, and I’d leave the framework where it is. The functions in core were written that way, which is why a Play controller could call them unchanged. If you add Cats Effect code to such an application, it can be joined to the futures: unsafeToFuture() turns an IO into a Future, and IO.fromFuture goes the other way.

23.9. Exercises

Solutions to the first two are in the book’s downloadable code, under ch23/exercises.

  1. Add GET /bookings/:name, which answers with one booking or a 404. In the routes file a parameter is written /bookings/:name and the method as find(name: String). findBooking in core does the looking up, and NotFound makes the result.

  2. Add a test for it to BookingControllerSpec: 200 for a name that has a booking and 404 for one that hasn’t.

  3. Give BookingBook a constructor parameter whose type is a trait with no implementation, such as trait Clock, and put @Inject() before the parameter list, importing Inject beside Singleton. Compile. Then run the tests and find Guice’s message in the output.

  4. In futures.scala, change val table to def table. Work out what it will print before running it.

Appendix A: Reading Scala 2

Scala 3 came out in 2021. A great deal of the Scala in production was written before that. Scala 2 code uses most of the same libraries, since they’re published for both versions. The older language is spelt differently, and one of its words, implicit, does several jobs that Scala 3 has given separate words, given, using and extension among them. This appendix shows one small program in both versions, so that you can read Scala 2 when you meet it.

The first version is written the way Scala 2.13 code usually is, 2.13 being the final version of Scala 2. Save it as desk.scala in a new directory called scala2. Its first line asks the scala command for a Scala 2 compiler, so it runs as it stands:

//> using scala 2.13.18
//> using jvm 25
package fullybooked

import java.time.{LocalTime => Time}
import scala.concurrent.duration._

sealed trait Status
object Status {
  case object Requested extends Status
  case object Confirmed extends Status
  final case class Cancelled(reason: String) extends Status
}

final case class Party(name: String, size: Int)

trait Display[A] {
  def text(value: A): String
}

object Display {
  implicit val statusDisplay: Display[Status] = new Display[Status] {
    def text(status: Status): String = status match {
      case Status.Requested         => "requested"
      case Status.Confirmed         => "confirmed"
      case Status.Cancelled(reason) => s"cancelled ($reason)"
    }
  }

  implicit def listDisplay[A](implicit each: Display[A]): Display[List[A]] =
    new Display[List[A]] {
      def text(values: List[A]): String = values.map(each.text).mkString(", ")
    }
}

object syntax {
  implicit class DisplayOps[A](value: A) {
    def shown(implicit display: Display[A]): String = display.text(value)
  }
}

class Tally {
  private var count = 0
  def add(size: Int): Unit = { count += size }
  def total: Int = count
}

object Desk {
  import syntax._

  def longest[A: Display](values: List[A]): String =
    values.map(implicitly[Display[A]].text).maxBy(_.length)

  def covers(sizes: Int*): Int = {
    val tally = new Tally
    sizes.foreach(tally.add)
    tally.total
  }

  def main(args: Array[String]): Unit = {
    val parties = List(Party("Hughes", 4), Party("Mbeki", 8))
    val statuses = List[Status](Status.Requested, Status.Cancelled("no deposit"))
    val eight = Time.of(20, 0)

    println(statuses.shown)
    println(longest(statuses))
    println(covers(parties.map(_.size): _*))
    for (party <- parties if party.size > 4) {
      println(s"${party.name} have the big table from $eight for ${90.minutes}")
    }
  }
}
$ scala run desk.scala
requested, cancelled (no deposit)
cancelled (no deposit)
12
Mbeki have the big table from 20:00 for 90 minutes

Here’s the same program in Scala 3. It goes in a second directory, scala3, under the same file name:

//> using scala 3.9.0
//> using jvm 25
package fullybooked

import java.time.LocalTime as Time
import scala.concurrent.duration.*

enum Status:
  case Requested, Confirmed
  case Cancelled(reason: String)

case class Party(name: String, size: Int)

trait Display[A]:
  def text(value: A): String

object Display:
  given Display[Status]:
    def text(status: Status): String = status match
      case Status.Requested => "requested"
      case Status.Confirmed => "confirmed"
      case Status.Cancelled(reason) => s"cancelled ($reason)"

  given [A] => (each: Display[A]) => Display[List[A]]:
    def text(values: List[A]): String = values.map(each.text).mkString(", ")

extension [A](value: A)(using display: Display[A])
  def shown: String = display.text(value)

class Tally:
  private var count = 0
  def add(size: Int): Unit = count += size
  def total: Int = count

def longest[A: Display](values: List[A]): String =
  values.map(summon[Display[A]].text).maxBy(_.length)

def covers(sizes: Int*): Int =
  val tally = Tally()
  sizes.foreach(tally.add)
  tally.total

@main def desk(): Unit =
  val parties = List(Party("Hughes", 4), Party("Mbeki", 8))
  val statuses = List[Status](Status.Requested, Status.Cancelled("no deposit"))
  val eight = Time.of(20, 0)

  println(statuses.shown)
  println(longest(statuses))
  println(covers(parties.map(_.size)*))
  for party <- parties if party.size > 4 do
    println(s"${party.name} have the big table from $eight for ${90.minutes}")

It prints the same four lines. The file is 52 lines long where the old one is 72.

A.1. Braces, new and main

In the old file the body of every class, trait and object is in braces, and so are the match and the longer method bodies. The party <- parties of the for is in parentheses and there’s no do. An if would be written if (free) seat else refuse, without then.

A class that isn’t a case class needs new to make an instance, as in Java, so covers says new Tally. The same word in new Display[Status] { ... } makes an instance of a trait on the spot, with the trait’s methods in the braces. In the Scala 3 file that’s a given Display[Status] with the method indented beneath it.

The program starts at a main method with Java’s signature, inside an object. The other common way was object Desk extends App, with the statements written straight into the object’s body. Scala 2 allowed only classes, traits and objects at the top level of a file, so longest and covers have to live inside Desk. Definitions meant for a whole package went in a package object.

A.2. An enum in five definitions

Scala 2 has no enum, so Status is a sealed trait, as Event was in chapter 8. Its cases sit in the companion object: a case object for each case without fields and a case class for the one with a reason. Both case classes in the file are marked final, which stops anything from extending them. Some Scala 2 codebases put final on every case class as a house rule.

A.3. The jobs of implicit

An implicit val or implicit def declares what Scala 3 calls a given. Scala 2 wanted names for them, so they’re called statusDisplay and listDisplay, though nothing in the file uses either name. The second has a parameter list of its own that starts with implicit. Scala 3 writes it as given [A] => (each: Display[A]) => Display[List[A]]: there’s a Display for a list whenever there’s one for its elements.

A parameter list that starts with implicit is a using clause: the compiler fills it in with a given of the right type. In longest, where the Display[A] has no name, implicitly[Display[A]] asks the compiler for it by hand. Scala 3’s word for that is summon.

An implicit class adds methods to a type that already exists, which is the job of extension in Scala 3. In the file, DisplayOps wraps any value so that shown can be called on it. Implicit classes are usually collected in an object called syntax or ops that has to be imported. Cats has an object of that kind behind the import cats.syntax.all.* we’ve been writing since chapter 11. The 90.minutes in main works because import scala.concurrent.duration._ brings in an implicit class for Int.

An implicit def with one ordinary parameter is an implicit conversion. The old file has none, but one such as implicit def toDuration(minutes: Int): FiniteDuration would let an Int be passed wherever a FiniteDuration is expected, and the call wouldn’t show it. When a value in old code seems to change type by itself, look for one of those in the imports. Scala 3 writes it as a given Conversion[Int, FiniteDuration] and warns when one is applied, unless the file that applies it has import scala.language.implicitConversions.

In Scala 2, import syntax._ brings in every implicit in the object along with everything else. In Scala 3 the * still brings in anything declared with implicit but leaves givens behind, and chapter 11’s import some.pkg.given fetches them.

A.4. Smaller differences

The Int* in the parameter list of covers means any number of Int arguments, like int... in Java. To pass a list where separate arguments are expected you mark it, with : _* in Scala 2 and * in Scala 3. The first import in the file gives LocalTime another name, Time.

Scala 2 Scala 3

import scala.concurrent.duration._

import scala.concurrent.duration.*

import java.time.{LocalTime => Time}

import java.time.LocalTime as Time

covers(sizes: _*)

covers(sizes*)

List[_], a list of something unknown

List[?]

class A extends B with C

class A extends B, C

def close() { ... }, a method that returns Unit

def close(): Unit = ...

A.5. Scala 3 reads most of it

In the scala2 directory, change the first line of desk.scala to ask for 3.9.0 and run it again:

$ scala run desk.scala
[warn] ./desk.scala:67:41
[warn] The syntax `x: _*` is no longer supported for vararg splices; use `x*` instead
[warn] This construct can be rewritten automatically under -rewrite -source 3.4-migration.
[warn]     println(covers(parties.map(_.size): _*))
[warn]                                         ^
requested, cancelled (no deposit)
cancelled (no deposit)
12
Mbeki have the big table from 20:00 for 90 minutes

The one warning is about the : _* on line 67. Scala 3.9 accepts braces, implicit and everything in the left-hand column of the table above except the last row, with a warning of this kind for List[_] as well. That lets a team change compiler first and restyle the code later. The compiler does some of the restyling when it’s given -rewrite. I ran both of these on the file:

$ scala compile desk.scala -O -rewrite -O -source:3.4-migration
$ scala compile desk.scala -O -rewrite -O -indent

Each -O hands the option after it to the compiler. The first command is the one the warning suggests, and it changed only line 67, where : _* became *. The second took the braces off every class, trait, object, match and multi-line body. It left implicit, new, the parentheses of the for and the old imports as they were.

Macros, code that a library runs inside the compiler, usually take more work to move than syntax does. Scala 2 had its own way of writing them, which Scala 3 replaced. A library built on the old macros has to be swapped for its Scala 3 version, and macros a team wrote itself have to be written again. The Scala 3 migration guide is at https://docs.scala-lang.org/scala3/guides/migration/planning.html.

Appendix B: The rest of the ecosystem

Part IV built its service from the Typelevel family of libraries, one of several that chapter 1 named. Here’s what the others are for, with a script to run wherever a few lines will show something, and then two places other than the JVM where Scala code can run. Each script goes in a directory of its own.

B.1. ZIO

ZIO is the other effect system in wide use. Save this as evening.scala and run it:

//> using scala 3.9.0
//> using jvm 25
//> using dep dev.zio::zio:2.1.27

import zio.*

object Evening extends ZIOAppDefault:
  def findTable(size: Int): Task[Int] =
    ZIO.sleep(100.millis).as(if size > 6 then 6 else 5)

  def run =
    for
      _ <- Console.printLine("Fully Booked is open")
      looking <- findTable(7).fork
      _ <- Console.printLine("looking for a table")
      table <- looking.join
      _ <- Console.printLine(s"table $table is free")
    yield ()
$ scala run evening.scala
Fully Booked is open
looking for a table
table 6 is free

A ZIO program is a value, as an IO is, and run chains five such programs in a for. Chapter 14’s start is called fork here, and join waits for the fiber and returns its result. ZIOAppDefault does the job of IOApp.

The nearest thing ZIO has to IO[Int] is Task[Int], a short name for one case of ZIO’s main type. That type has three parameters where IO has one: a ZIO[R, E, A] needs an R from its surroundings before it can run, can fail with an E and produces an A. The R is for things like a database or the configuration, which ZIO supplies through values it calls layers, where we passed function arguments and Play used Guice. The E does what our code did by keeping an Either inside the IO. A Task needs nothing and can fail with any Throwable.

ZIO has libraries of its own for most of what Parts III and IV used, ZIO HTTP, ZIO JSON, ZIO Streams and a test framework among them. Tapir has a ZIO server, and an interop library lets ZIO code use Cats Effect libraries such as doobie. Since ZIO and Cats Effect do the same job, I’d pick the one the team already knows and keep a service to one of them.

B.2. The Li Haoyi libraries

Li Haoyi’s libraries have no effect type at all. A call blocks until it has an answer, returns a plain value and throws if it fails. Save this one as parties.scala:

//> using scala 3.9.0
//> using jvm 25
//> using toolkit default
//> using javaOpt --sun-misc-unsafe-memory-access=allow

import upickle.default.{read, write, ReadWriter}

case class Party(name: String, size: Int) derives ReadWriter

@main def parties(): Unit =
  val file = os.temp.dir() / "parties.json"
  os.write(file, write(List(Party("Hughes", 4), Party("Mbeki", 8))))
  println(os.read(file))
  println(read[List[Party]](os.read(file)).map(_.size).sum)
$ scala run parties.scala
[{"name":"Hughes","size":4},{"name":"Mbeki","size":8}]
12

The script uses two of Li Haoyi’s libraries. OS-Lib, which the code calls os, is for files and processes, and uPickle turns case classes into JSON and back. Neither appears in a dep line, because using toolkit brings in the Scala Toolkit, a small set of libraries the Scala project recommends for everyday jobs. It holds OS-Lib and uPickle, along with sttp for HTTP requests and MUnit. Li Haoyi also wrote Requests for HTTP calls, a small web framework called Cask, and Mill, a build tool that some teams use in place of sbt.

I’d use this style for a script or a command-line tool. Li Haoyi’s book, Hands-on Scala Programming, teaches the language through these libraries.

B.3. Pekko

Pekko, the open-source fork of Akka, is what Play 3 runs on. On its own it’s a toolkit for building a program out of actors, objects that communicate only by sending messages. An actor keeps its state to itself and handles one message at a time, so that state needs no lock. On top of actors Pekko has streams, a way of spreading actors across a cluster of machines, and persistence for their state.

Pekko suits systems that keep a great deal of live state in memory across many machines. A service like Fully Booked, whose state is in a database, doesn’t need it, and I wouldn’t add actors to one. If you inherit a codebase with akka or org.apache.pekko imports outside the Play parts, start reading at receive and onMessage, the methods where an actor handles its messages.

B.4. Spark

Much of the paid work in Scala is data engineering on Apache Spark. Spark is an engine for processing large amounts of data on a cluster, and it’s written in Scala itself. A Spark job calls methods such as map and filter on a Dataset, a collection that is spread over many machines. Spark decides where each step runs.

Spark 4 is built for Scala 2.13 and has no Scala 3 build, so Spark code is normally written in the Scala 2 that appendix A describes. Spark can be driven from Python as well, and often is.

B.5. Scala.js and Scala Native

Everything in this book has run on the JVM, but the Scala compiler has two other targets, JavaScript and native executables. Save this as tables.scala:

//> using scala 3.9.0

case class Table(number: Int, seats: Int)

val tables = List(Table(1, 2), Table(3, 4), Table(5, 6), Table(6, 8))

def smallestFor(size: Int): String =
  tables.filter(_.seats >= size).minByOption(_.seats) match
    case Some(table) => s"table ${table.number} for a party of $size"
    case None => s"nothing seats $size"

@main def seat(): Unit =
  println(smallestFor(5))
  println(smallestFor(9))

As usual, scala run tables.scala runs it on the JVM. With Node.js installed, adding --js compiles it to JavaScript and runs that:

$ scala run tables.scala --js
table 5 for a party of 5
nothing seats 9

With the clang compiler installed, --native makes a native executable and runs it. The command prints some twenty lines about compiling and linking first, which I’ve left out:

$ scala run tables.scala --native
table 5 for a party of 5
nothing seats 9

Scala.js is mainly for the browser. A team can write the front end in the language of the back end and share code between the two, such as validation rules or a Tapir endpoint description. Scala Native compiles to a program that needs no JVM and starts at once, which suits command-line tools. To keep the executable, run scala --power package tables.scala --native -o seat, which writes it to a file called seat. On my machine that file is 2.5 megabytes and runs in under a hundredth of a second. The --power is there because the scala command treats package as a restricted feature.

Neither target can use a Java library, since there’s no JVM underneath. Scala libraries have to be published for each target separately. Cats, Cats Effect and fs2 are published for both.