21. Testing the service

Chapter 6 put off a test for the last-table race, and chapter 20’s SQL has only ever been run by hand. Both get tests in this chapter, and one of the new tests turns up a bug that has been in the booking rules since chapter 4.

The chapter needs five new libraries, all for tests only. In build.sbt, add one line to the list for core, with a comma after the munit above it:

      "org.scalameta" %% "munit-scalacheck" % "1.3.1" % Test

and four to the list for service, again with a comma after its munit:

      "org.typelevel" %% "munit-cats-effect" % "2.2.1" % Test,
      "org.typelevel" %% "cats-effect-testkit" % "3.7.1" % Test,
      "com.dimafeng" %% "testcontainers-scala-munit" % "0.44.1" % Test,
      "com.dimafeng" %% "testcontainers-scala-postgresql" % "0.44.1" % Test

21.1. Tests that return IO

Our tests of an IO have ended in unsafeRunSync() since chapter 13, and needed an import to supply the threads. With munit-cats-effect a test hands its IO back and the suite runs it. This chapter’s test files for the service all go in service/src/test/scala. Replace api.test.scala there with this version:

package fullybooked

import cats.effect.IO
import cats.syntax.all.*
import sttp.model.StatusCode

class ApiTests extends munit.CatsEffectSuite:
  def answers(requests: List[NewBooking]): IO[List[Either[StatusCode, String]]] =
    for
      bookings <- Bookings.starting(regulars)
      results <- requests.traverse(request => create(bookings, request))
    yield results.map(result => result.left.map(refused => refused(0)).map(view => view.name))

  test("a good request is booked"):
    assertIO(answers(List(NewBooking("Ibrahim", 7, "20:00", None))), List(Right("Ibrahim")))

  test("a request that fails validation is a 400 listing every problem"):
    val bad = NewBooking("", 7, "8pm", Some("ask for Dev"))
    for
      bookings <- Bookings.starting(regulars)
      result <- create(bookings, bad)
    yield assertEquals(
      result.left.map(refused => (refused(0), refused(1).problems.size)),
      Left((StatusCode.BadRequest, 3))
    )

  test("the second party after the last table gets a 409"):
    val both = List(
      NewBooking("Ibrahim", 7, "20:00", None),
      NewBooking("Castellanos", 8, "20:00", None)
    )
    assertIO(answers(both), List(Right("Ibrahim"), Left(StatusCode.Conflict)))

  test("a name with no booking is a 404"):
    val program = Bookings.starting(regulars).flatMap(bookings => find(bookings, "Smith"))
    assertIO(program.map(_.left.map(refused => refused(0))), Left(StatusCode.NotFound))
$ sbt "service/test fullybooked.ApiTests"
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] elapsed time: 2 s

The suite extends munit.CatsEffectSuite, whose assertIO runs a program and compares its result with a value. A test can also be a for that ends in ordinary assertions, like the second one here. Either way the body’s type is IO, so a test that forgets to run something no longer passes by accident. The other suites can be converted by changing the class they extend, returning the IO from each test (usually through assertIO) and deleting the unsafeRunSync() calls with their import. I’d do each suite the next time I had a reason to open it.

21.2. Time under control

Chapter 15’s retrying waits between attempts. confirm allows three attempts and gives the gateway a fifth of a second to answer each time. The tests for retrying used pauses of a millisecond to stay quick, and nothing tested the timings in confirm at all. Create time.test.scala:

package fullybooked

import cats.effect.IO
import cats.effect.testkit.TestControl
import java.io.IOException
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import scala.concurrent.duration.*

class TimeTests extends munit.CatsEffectSuite:
  def timed[A](program: IO[A]): IO[FiniteDuration] =
    for
      start <- IO.monotonic
      _ <- program.attempt
      end <- IO.monotonic
    yield end - start

  test("an hour's sleep takes no real time"):
    assertIO(TestControl.executeEmbed(timed(IO.sleep(1.hour))), 1.hour)

  test("retrying pauses for 20, 40 and 80 milliseconds before it gives up"):
    val down = IO.raiseError[Unit](IOException("gateway unavailable"))
    val program = timed(retrying(down, attempts = 4, pause = 20.millis))
    assertIO(TestControl.executeEmbed(program), 140.millis)

  test("a gateway that never answers takes three timeouts and two pauses"):
    val logger: Logger[IO] = Slf4jLogger.getLogger[IO]
    val phone = Phone.parse("07700 900164")
    val booking = Booking(Party("Moreau", 3, phone), Table(3, 4), eight, Status.Confirmed)
    val program =
      for
        calls <- IO.ref(Map[Phone, Int]())
        gateway = FlakyGateway(calls, failsTwice = Set(), hangs = phone.toSet)
        elapsed <- timed(confirm(gateway, logger, booking))
        made <- calls.get
      yield (elapsed, made.values.sum)
    assertIO(TestControl.executeEmbed(program), (660.millis, 3))
$ sbt "service/test fullybooked.TimeTests"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

A test can now sleep for an hour and take no real time, as the first one does. TestControl.executeEmbed runs a program on a runtime where TestControl is in charge of the clock. When every fiber is asleep it moves the clock to the next moment anything is due to wake, so no real time passes. The helper timed measures how long a program took by reading that clock with IO.monotonic.

Because the test’s clock only moves when its fibers sleep, a test can assert exact times. Four attempts with a pause that starts at 20 milliseconds and doubles should wait for 20, 40 and 80. The test of retrying checks that the total is 140. The last test puts the Moreaus’ number behind a gateway that hangs. Three timeouts of 200 milliseconds and pauses of 20 and 40 come to 660, after three calls. The WARN line that test prints is confirm reporting that the gateway didn’t answer.

21.3. Properties

Our tests so far have named their inputs: a party of four at eight, a party of nine. A property test states something that should hold for any input and lets a library, here ScalaCheck, look for a counter-example. Create rules.test.scala in core/src/test/scala:

package fullybooked

import java.time.LocalTime
import org.scalacheck.Gen
import org.scalacheck.Prop.forAll

class RuleProperties extends munit.ScalaCheckSuite:
  case class Asked(size: Int, time: LocalTime)

  val request: Gen[Asked] =
    for
      size <- Gen.choose(1, 10)
      quarter <- Gen.choose(0, 95)
    yield Asked(size, LocalTime.MIDNIGHT.plusMinutes(quarter * 15L))

  def book(requests: List[Asked]): List[Booking] =
    requests.foldLeft(List[Booking]()): (bookings, asked) =>
      decide(bookings, Party("Guest", asked.size), asked.time) match
        case Right(booking) => bookings :+ booking
        case Left(_) => bookings

  def minutes(time: LocalTime): Int = time.getHour * 60 + time.getMinute

  def clash(a: Booking, b: Booking): Boolean =
    a.table == b.table &&
      minutes(a.time) < minutes(b.time) + b.party.stayMinutes &&
      minutes(b.time) < minutes(a.time) + a.party.stayMinutes

  property("no two parties are given the same table at the same time"):
    forAll(Gen.listOf(request)): requests =>
      val booked = book(requests)
      val clashes = for
        (a, i) <- booked.zipWithIndex
        b <- booked.drop(i + 1)
        if clash(a, b)
      yield (a.time, b.time, a.table.number)
      assertEquals(clashes, List())

A Gen describes how to make random values. The one called request makes an Asked, a party of one to ten at any quarter of an hour in the day. It’s built by combining two smaller generators in a for.

The property is checked against whole lists of requests, which Gen.listOf(request) generates and forAll passes to the block beneath it. The block puts a list through decide one request after another with book, then looks for two of the resulting bookings that share a table at overlapping times. zipWithIndex numbers the bookings so that each pair is looked at once. The overlap check, clash, works in minutes since midnight and shares no code with freeFor, so a mistake in freeFor is unlikely to be repeated in the test.

$ sbt "core/test fullybooked.RuleProperties"
Falsified after 26 passed tests.
> ARG_0: List("Asked(8,23:45)", "Asked(7,22:15)")
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0

ScalaCheck tries a hundred random lists, and this time the twenty-seventh failed. It then shrank that list, dropping requests for as long as the property still failed, which left two. ScalaCheck shrinks numbers as well, and in a plain tuple it could have taken the party size below one. A request is an Asked because ScalaCheck has no way to shrink a class of ours. Yours will probably be a different pair, because the lists are random. Mine is a party of eight at quarter to midnight, then a party of seven at quarter past ten, and book gives both of them table 6.

The party of eight is due to leave two hours later, at quarter to two. LocalTime has no days in it, so leaveBy is 01:45. Since that’s before 22:15, freeFor concludes that the table is free. A stay that runs past midnight hadn’t occurred to me, so I never wrote a test for one. Over HTTP the same bug gives a 500 for a POST at 22:45, because Postgres won’t build a range that ends before it starts. The restaurant stops taking orders at half past nine, but that rule is in whenOpen from chapter 7, which decide never calls.

To fix it, decide has to refuse a time when the kitchen is closed. If you did the first exercise in chapter 9, you have the error already. Otherwise add case KitchenClosed(time: LocalTime) to the end of BookingError in errors.scala. Compile, and -Werror points at the next thing to change: explain no longer covers every case. Add this to the end of it:

  case BookingError.KitchenClosed(time) => s"the kitchen is closed at $time"

Then, in decide in availability.scala, put a second check between the first and the else:

  if party.size > Booking.largestParty then Left(BookingError.PartyTooLarge(party.size))
  else if time.isBefore(opens) || time.isAfter(lastOrders) then
    Left(BookingError.KitchenClosed(time))

In routes.scala, create answers 409 for anything reserve refuses, but a closed kitchen is the client’s mistake and should be 400. A party too large for any table should be 400 too. Add this function above find, and change the last line of create to call it in place of StatusCode.Conflict. If you wrote statusFor in chapter 19’s exercises, give that the new case and keep it instead:

def statusOf(error: BookingError): StatusCode = error match
  case BookingError.NothingFree(_, _) | BookingError.NotAllowed(_, _) => StatusCode.Conflict
  case BookingError.PartyTooLarge(_) | BookingError.KitchenClosed(_) | BookingError.NameMissing |
      BookingError.BadPartySize(_) | BookingError.BadTime(_) | BookingError.BadPhone(_) =>
    StatusCode.BadRequest

With the bug fixed, add two more properties to the end of RuleProperties. The second one’s input is Gen.listOfN(11, Gen.numChar), a list of eleven random digits:

  property("nobody is seated at a table too small for them"):
    forAll(Gen.listOf(request)): requests =>
      assert(book(requests).forall(booking => booking.party.size <= booking.table.seats))

  property("a phone number survives being parsed and printed"):
    forAll(Gen.listOfN(11, Gen.numChar)): digits =>
      val typed = digits.mkString
      assertEquals(Phone.parse(typed).map(_.formatted.filterNot(_.isWhitespace)), Some(typed))
$ sbt "core/test fullybooked.RuleProperties"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

Add a test for the new answer to the end of ApiTests as well:

  test("a booking for half past eleven at night is a 400"):
    assertIO(
      answers(List(NewBooking("Night", 2, "23:30", None))),
      List(Left(StatusCode.BadRequest))
    )

Properties suit rules that can be said in a sentence and broken by inputs I wouldn’t think to type, which describes most of availability.scala. When one fails, the output includes a seed and a line to add to the suite. With that line in, ScalaCheck replays the same case while you fix the bug.

21.4. A real Postgres

Nothing tests the SQL yet, or the constraint, which only exists inside a database. Testcontainers starts a real Postgres in Docker for the length of a suite. Create postgres.test.scala:

package fullybooked

import cats.effect.IO
import cats.syntax.all.*
import com.dimafeng.testcontainers.PostgreSQLContainer
import com.dimafeng.testcontainers.munit.TestContainerForAll
import doobie.implicits.*
import org.testcontainers.utility.DockerImageName

class PostgresTests extends munit.CatsEffectSuite, TestContainerForAll:
  override val containerDef: PostgreSQLContainer.Def =
    PostgreSQLContainer.Def(dockerImageName = DockerImageName.parse("postgres:18"))

  def withBookings[A](use: PostgresBookings => IO[A]): IO[A] =
    withContainers: postgres =>
      val config = DatabaseConfig(postgres.jdbcUrl, postgres.username, postgres.password)
      migrate(config) >> transactor(config).use: xa =>
        val seed =
          sql"TRUNCATE bookings".update.run >> regulars.traverse_(booking => insert(booking).run)
        val busy = sql"SELECT 1 FROM pg_sleep(0.1)".query[Int].unique.transact(xa)
        seed.transact(xa) >> List.fill(8)(busy).parSequence_ >> use(PostgresBookings(xa))

  test("a booking comes back from the database as it went in"):
    withBookings: bookings =>
      for
        made <- bookings.reserve(Party("Ibrahim", 7, Phone.parse("07700 900141")), eight)
        stored <- bookings.all
      yield
        assert(made.isRight)
        assertEquals(stored.filter(_.party.name == "Ibrahim"), made.toList)

  test("twenty parties after the last table: one gets it"):
    withBookings: bookings =>
      val parties = List.tabulate(20)(n => Party(s"Party $n", 7))
      for
        answers <- parties.parTraverse(party => bookings.reserve(party, eight))
        stored <- bookings.all
      yield
        assertEquals(answers.count(_.isRight), 1)
        assertEquals(stored.count(_.table.number == 6), 1)
$ sbt "service/test fullybooked.PostgresTests"
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] elapsed time: 2 s

Extending TestContainerForAll as well as CatsEffectSuite starts a container before the suite’s first test and removes it after the last. The container comes from the image that containerDef names. Docker gives the container a port chosen at random on your machine, so it can’t collide with the database from chapter 20. withContainers passes the running container to our code, with its address and password. Our fixture, withBookings, runs the migrations against the container, opens a pool, empties the table and seats the regulars, so each test starts from the same evening.

A new pool opens its connections in the background. The fixture waits for them before it hands over the PostgresBookings, by keeping all eight busy for a tenth of a second. Each runs a query that returns the one row unique expects. Before I added that step the second test passed with the constraint taken away, which means its twenty reservations weren’t overlapping.

One test checks that a booking read back from the table equals the one reserve returned, using made.toList to turn the Either into a list of one booking. The other is the last-table race. Twenty parties go for table 6 at once through a real connection pool, and exactly one of them gets it. I checked that this test fails when it should. Without ON CONFLICT DO NOTHING on the insert it fails with the constraint’s error. With the second migration taken away, eight parties are told the table is theirs. If you try the same, run the tests with sbt service/testFull, because test doesn’t notice that a migration file has gone and says there are no tests to run.

The whole suite takes two or three seconds on my machine, about half of it Postgres starting, with twenty-odd lines of Testcontainers’ own logging on the way. It does need Docker, on the build server as well as on yours.

21.5. The service without a network

The tests in ApiTests call create directly, so nothing yet checks that the JSON has the field names a client expects. Nor does anything check that a Refusal turns into the status code it carries. We can check both without opening a port, because an http4s application is a function from a request to an IO of a response.

At the moment Server builds its application inside serve, where a test can’t reach it. Replace server.scala with a version that builds it in a function of its own, httpApp:

package fullybooked

import cats.effect.{IO, IOApp}
import com.comcast.ip4s.{ipv4, Port}
import org.http4s.HttpApp
import org.http4s.ember.server.EmberServerBuilder
import org.http4s.server.middleware
import org.typelevel.log4cats.Logger
import org.typelevel.log4cats.slf4j.Slf4jLogger
import sttp.tapir.server.http4s.Http4sServerInterpreter
import sttp.tapir.swagger.bundle.SwaggerInterpreter

def httpApp(title: String, bookings: Bookings): HttpApp[IO] =
  val routes = bookingRoutes(bookings)
  val docs = SwaggerInterpreter().fromServerEndpoints[IO](routes, title, "0.1.0")
  Http4sServerInterpreter[IO]().toRoutes(routes ++ docs).orNotFound

object Server extends IOApp.Simple:
  val logger: Logger[IO] = Slf4jLogger.getLogger[IO]

  def run: IO[Unit] =
    for
      config <- loadConfig
      applied <- migrate(config.database)
      _ <- logger.info(s"$applied migrations applied")
      port <- portFrom(config)
      _ <- transactor(config.database).use(xa => serve(config, port, PostgresBookings(xa)))
    yield ()

  def portFrom(config: AppConfig): IO[Port] =
    IO.fromOption(Port.fromInt(config.port))(
      IllegalArgumentException(s"${config.port} isn't a port")
    )

  def serve(config: AppConfig, port: Port, bookings: Bookings): IO[Unit] =
    EmberServerBuilder
      .default[IO]
      .withHost(ipv4"0.0.0.0")
      .withPort(port)
      .withHttpApp(
        middleware.Logger.httpApp(logHeaders = false, logBody = false)(
          httpApp(config.restaurant, bookings)
        )
      )
      .build
      .useForever

Then create http.test.scala:

package fullybooked

import cats.effect.IO
import org.http4s.{MediaType, Method, Request}
import org.http4s.headers.`Content-Type`
import org.http4s.implicits.*

class HttpTests extends munit.CatsEffectSuite:
  def post(body: String): Request[IO] =
    Request[IO](Method.POST, uri"/bookings")
      .withEntity(body)
      .withContentType(`Content-Type`(MediaType.application.json))

  test("a booking made over HTTP can be read back over HTTP"):
    for
      bookings <- Bookings.starting(List())
      app = httpApp("Fully Booked", bookings)
      created <- app.run(post("""{"name":"Hughes","size":4,"time":"18:30"}"""))
      listed <- app.run(Request[IO](Method.GET, uri"/bookings"))
      body <- listed.as[String]
    yield
      assertEquals(created.status.code, 201)
      assertEquals(
        body,
        """[{"name":"Hughes","size":4,"table":3,"time":"18:30","status":"confirmed"}]"""
      )

  test("the kitchen's hours reach the client as a 400"):
    for
      bookings <- Bookings.starting(List())
      response <- httpApp("Fully Booked", bookings).run(
        post("""{"name":"Night","size":2,"time":"23:30"}""")
      )
      body <- response.as[String]
    yield
      assertEquals(response.status.code, 400)
      assertEquals(body, """{"problems":["the kitchen is closed at 23:30"]}""")

  test("a path nobody serves is a 404"):
    for
      bookings <- Bookings.starting(List())
      response <- httpApp("Fully Booked", bookings).run(Request[IO](Method.GET, uri"/menu"))
    yield assertEquals(response.status.code, 404)
$ sbt "service/test fullybooked.HttpTests"
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
[success] elapsed time: 2 s

Calling app.run(request) sends a request through Tapir’s decoding, our logic and the encoding of the answer without touching a socket. With the in-memory Bookings behind the application, these tests need no database either. post builds the request and gives it a body with withEntity. The header’s name is in backticks because that’s how Scala writes an identifier with a hyphen in it. uri"/bookings" is another literal that’s checked at compile time. The tests read each response’s body with as[String]. They compare the status as a number, because the name Status would mean our enum here and http4s has a type called Status too.

I don’t write the same number of each kind of test. Most of mine are plain tests of functions, because that’s where the rules are. A handful of properties go with the rules where an input I hadn’t thought of could break something. A few tests go through HTTP to pin down the JSON and the status codes, and a few go to a real database for the SQL and for whatever only the database enforces. Run them all:

$ sbt testFull
[info] Passed: Total 34, Failed 0, Errors 0, Passed 34
[info] Passed: Total 38, Failed 0, Errors 0, Passed 38
[info] Passed: Total 6, Failed 0, Errors 0, Passed 6
[success] elapsed time: 2 s

21.6. Exercises

Solutions are in the book’s downloadable code, under ch21/exercises.

  1. Rewrite the first two tests of ConfirmationTests, the ones about retrying, in a CatsEffectSuite, with pauses of a minute under TestControl. interceptMessageIO[IOException]("...")(program) passes when the program fails with that exception and message.

  2. Write a property saying that an empty restaurant, at any quarter of an hour between opens and lastOrders, gives a party of one to eight the smallest table that fits.

  3. Insert a row with a status of 'Vanished' straight into the table with sql, and test that all on a PostgresBookings then fails with the message from toBooking. withBookings keeps the transactor to itself, so this test needs withContainers and a transactor of its own.

  4. Test GET /bookings/Smith through httpApp, checking the status and the body.