Today we’ll make a simple web app, and try to decipher what nuances and tradeoffs exist between the available solutions. So let’s dive in and see if one stands out as more applicable to our needs than the others. Ideally we’ll end the day with examples of apps that call into our database and returns some HTML, but let’s see how that ambition fares.
Table of Contents
Nothing At All
Cowboy
Plugged Cowboy
Bandit
Connecting To Database
Phoenix Framework
What Phoenix Gives Us
Conclusion
Nothing At All
What if we… don’t choose a solution? It is technically possible to serve HTML directly with no dependencies at all by using the Erlang TCP/IP socket module, what would that look like? Not that this will be viable solution, but it might be illuminating to see what it takes to work without abstractions.
We’ll basically need to listen on a TCP socket and wait for traffic, and then throw data down the wire when a request arrives. Here’s a script that does that:
def start(port) do
# Listen on a TCP socket on the specified port
# :binary – Treat data as raw binary, instead of
# being automatically converted into
# Elixir strings (which are UTF-8 encoded).
# It’d be unnecessary to convert, as the
# HTTP protocol uses raw bytes.
# packet: :line – Frame messages using newline delimiters,
# which is the expected shape of HTTP-data
# active: false – Require manual fetching of messages. In
# Erlang, active mode controls the
# automatic sending of messages to the
# socket’s controlling process. We disable
# this behavior, so our server can control
# when and how it reads data
{:ok, socket} = :gen_tcp.listen(port, [
:binary, packet: :line, active: false
])
IO.puts(“Listening on port #{port}“)
loop_handle_client_connection(socket)
end
defp loop_handle_client_connection(socket) do
# Wait for a new client connection. This is a blocking call
# that waits until a new connection arrives.
# A connection returns a `client_socket` which is connected
# to the client, so we can send a reply back.
{:ok, client_socket} = :gen_tcp.accept(socket)
send_hello_world_response(client_socket)
:gen_tcp.close(client_socket)
# Recursively wait for the next client connection
loop_handle_client_connection(socket)
end
defp send_hello_world_response(client_socket) do
# Simple HTML content for the response.
content = “<h1>Hello, World!</h1>”
# Generate the entire raw HTTP response, which includes
# calculating content-length header.
response = “””
HTTP/1.1 200 OK
content-length: #{byte_size(content)}
content-type: text/html
#{content}
“””
:gen_tcp.send(client_socket, response)
end
end
SimpleServer.start(8080)
It’s not exactly easy code to read because it deals with such low-level concepts (hence all the comments to clarify what the various commands do 😅). But it does let us start the script in one terminal and probe it with curl in another, and actually see HTML getting returned:
Listening on port 8080 | <h1>Hello, World!</h1>%
So that works, if by “works” we mean this is technically returning HTML.
Is it actually useful? Not in any practical sense no, it’d be quite an adventure to properly implement all the features missing from this! But it hints at the hard work that gets solved by the libraries we’ll dive into next; it’s no exaggeration to say we stand on the shoulders of giants.
ℹ️ BTW the full code to this section is here.
Cowboy
Cowboy is a classic choice in Elixir: A small, minimalist HTTP server that’s proven itself for over a decade. It’s actually an Erlang library, the main language of the BEAM VM, but it’s worth including here because it’s used in so many Elixir projects.
To try out Cowboy we first need to scaffold an Elixir project, and there’s a straightforward to do that with mix, the Elixir build tool:
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/cowboy_example.ex
* creating test
* creating test/test_helper.exs
* creating test/cowboy_example_test.exs
Your Mix project was created successfully.
You can use “mix” to compile it, test it, and more:
cd cowboy_example
mix test
Run “mix help” for more commands.
And then we add Cowboy as a dependency and install dependencies:
$ git-nice-diff -U1
cowboy_example/mix.exs
L#23:
[
+ {:cowboy, “~> 2.11”}
# {:dep_from_hexpm, “~> 0.3.0”},
$ mix deps.get
Resolving Hex dependencies…
Resolution completed in 0.043s
New:
cowboy 2.12.0
cowlib 2.13.0
ranch 1.8.0
* Getting cowboy (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run
`mix hex.sponsor` to learn more
ℹ️ BTW git-nice-diff is just a small script that works like git diff but simplifies the output to make it easier to show diffs in this article. You can find it here if you’re curious.
And then we need to write some code that actually configures and starts the Cowboy server:
def start_server do
# Set up the routing table for the Cowboy server, so root
#requests (“/”) direct to our handler.
dispatch = :cowboy_router.compile([{:_, [
{“/”, CowboyExample.HelloWorldHandler, []}
]}])
# Start the Cowboy server in “clear mode” aka plain HTTP
# options – Configuration options for the server itself
# (this also supports which IP to bind to,
# SSL details, etc.)
# `env` – Configuration map for how the server
# handles HTTP requests
# (this also allows configuring timeouts,
# compression settings, etc.)
{:ok, _} =
:cowboy.start_clear(
:my_name,
[{:port, 8080}],
%{env: %{dispatch: dispatch}}
)
IO.puts(“Cowboy server started on port 8080”)
end
end
defmodule CowboyExample.HelloWorldHandler do
# `init/2` is the entry point for handling a new HTTP request
# in Cowboy
def init(req, _opts) do
req = :cowboy_req.reply(200, %{
“content-type” => “text/html”
}, “<h1>Hello World!</h1>”, req)
# Return `{:ok, req, state}` where `state` is
# handler-specific state data; here, it’s `:nostate`
# as we do not maintain any state between requests.
{:ok, req, :nostate}
end
end
Previously in the “Nothing At All” example we ran our code as a script via elixir <script.exs>, but this time we’ll take a small step towards a more common approach for applications where we start an IEx (Interactive Elixir) shell. From that we manually (for now) start our server:
Generated cowboy_example app. | <h1>Hello World!</h1>%
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] |
|
Interactive Elixir (1.16.1) – |
press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> CowboyExample.start_s |
erver |
Cowboy server started on port |
8080 |
:ok |
iex(2)> |
🎉 Hooray, we (once again) have an incredibly basic webserver. We’ve definitely moved up an abstraction level: No more dealing directly with sockets, the Cowboy server handles all that and “just” invokes our request/response handler function.
I still don’t find this easy to read, maybe mostly because the Erlang syntax adds an extra layer of mental translation. Not only in the code itself but also because Cowboy’s documentation is all Erlang-centric and so we have to understand how to map its examples into Elixir to make it all work. But at least the server works, and that’s a pretty good step!
ℹ️ BTW the full code to this section is here.
Plugged Cowboy
The previous example used plain Cowboy, but actually Cowboy seems to be more commonly used via a dependency called Plug, which is an Elixir library that that makes it easy to write html-responding functions. To use Plug with Cowboy the Plug documentation suggests using a library called plug_cowboy which combines both (it’s an adapter library that lets Plug control the Cowboy server). Let’s see how that looks!
First, we need a new project. But this time let’s do it so our server starts automatically when opening IEx. The standard solution for this is to generate an “OTP application“, which includes a main supervisor that will take care of our server process. We could create those files ourselves, but mix has a –sup flag that does it for us:
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/plugged_cowboy.ex
* creating lib/plugged_cowboy/application.ex
* creating test
* creating test/test_helper.exs
* creating test/plugged_cowboy_test.exs
Your Mix project was created successfully.
You can use “mix” to compile it, test it, and more:
cd plugged_cowboy
mix test
Run “mix help” for more commands.
And then we add our dependency:
$ git-nice-diff -U1
/plugged_cowboy/mix.exs
L#24:
[
+ {:plug_cowboy, “~> 2.0”}
# {:dep_from_hexpm, “~> 0.3.0”},
$ mix deps.get
Resolving Hex dependencies…
Resolution completed in 0.104s
New:
cowboy 2.10.0
cowboy_telemetry 0.4.0
cowlib 2.12.1
mime 2.0.5
plug 1.15.3
plug_cowboy 2.7.0
plug_crypto 2.0.0
ranch 1.8.0
telemetry 1.2.1
* Getting plug_cowboy (Hex package)
* Getting cowboy (Hex package)
* Getting cowboy_telemetry (Hex package)
* Getting plug (Hex package)
* Getting mime (Hex package)
* Getting plug_crypto (Hex package)
* Getting telemetry (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run `mix hex.sponsor` to learn more
Then it’s time to write our request-handler plug, and we get to see why the Plug library is so widely used: It has nice and simple self-describing functions that makes it straightforward to generate a proper HTML response:
import Plug.Conn
# Plugs must define `init/1`, but we have nothing to configure so it’s just a no-op implementation
def init(options), do: options
# `call/2` is the main function of a Plug, and is expected to process the request and generate a response
def call(conn, _options) do
conn
# Both functions below are part of `Plug.Conn`s functions, they’re available because we imported it
|> put_resp_content_type(“text/plain”)
|> send_resp(200, “Hello, World!”)
end
end
And finally we configure and register Plug.Cowboy to use our plug:
/plugged_cowboy/lib/plugged_cowboy/application.ex
L#10:
children = [
+ # Connect Plug.Cowboy plug handler
+ {Plug.Cowboy, plug: PluggedCowboy.MyPlug, scheme: :http, options: [port: 8080]}
# Starts a worker by calling: PluggedCowboy.Worker.start_link(arg)
And… that all just works:
Erlang/OTP 26 [erts-14.2.2] [ | Hello, World!%
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] |
Interactive Elixir (1.16.1) – |
press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
This plug-variant of Cowboy has quite nicely simplified what it takes to start the server, which is exactly why Plug exists. And we even got our code more aligned to normal idiomatic Elixir 👌
ℹ️ BTW the full code to this section is here.
Bandit
This project has a funny name, because it’s an alternative to Cowboy: Bandit is a webserver written in pure Elixir, and it has an internal design that more naturally integrates with the underlying Erlang concurrent programming model which results in greater performance.
ℹ️ BTW the naming-fun goes even deeper: Cowboy pulls in Ranch as a dependency, and so Bandit pulls in… Thousand Island 😂 (non-native English speakers might have to look up the dictionary definitions to follow these puns).
Like before: New project, then add and get our dependencies. And we’ll need to pull in Plug directly this time, because Bandit is already designed to be used with it:
$ cd bandit_example
$ git-nice-diff -U1
/bandit_example/mix.exs
L#24:
[
+ {:bandit, “~> 1.0”},
+ {:plug, “~> 1.0”}
# {:dep_from_hexpm, “~> 0.3.0”},
$ mix deps.get
ℹ️ BTW the > /dev/null just silences the mix new command, so we don’t have to cover as much output.
And then we write… pretty much exactly the same code as we did for Plugged Cowboy:
/bandit_example/lib/bandit_example/application.ex
L#10:
children = [
+ {Bandit, plug: BanditExample.MyPlug, port: 8080}
# Starts a worker by calling: BanditExample.Worker.start_link(arg)
/bandit_example/lib/bandit_example/my_plug.ex
L#1:
+defmodule BanditExample.MyPlug do
+ import Plug.Conn
+
+ def init(options), do: options
+
+ def call(conn, _options) do
+ conn
+ |> put_resp_content_type(“text/plain”)
+ |> send_resp(200, “Hello, World!”)
+ end
+end
And that just works:
| Hello, World!%
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] |
|
Interactive Elixir (1.16.1) – |
press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
Bandit ends up being almost a drop-in replacement for Cowboy, and that’s by design because Plug is really the primary developer-facing part of this sever-equation. So that’s a pretty cool separation of concerns there, Elixir community. Well done 👍
But this time, could we go one step further and connect to our database?
Connecting To Database
First we need a way for Elixir to connect to our database, and Elixir’s Postgres database adapter is called Postgrex. So let’s download that:
bandit_example/mix.exs
L#25:
{:bandit, “~> 1.0”},
– {:plug, “~> 1.0”}
+ {:plug, “~> 1.0”},
+ {:postgrex, “>= 0.0.0”}
# {:dep_from_hexpm, “~> 0.3.0”},
$ mix deps.get > /dev/null
And the second step to connecting to a database is to start that database 😅. We need to store its files somewhere, and it’s an Elixir convention (actually Erlang, but, same point) to put them in priv/db because that’s a folder meant for private, static files that are “part of the application but aren’t source code”:
$ initdb -D priv/db
$ pg_ctl -D priv/db -l logfile start
And we also need a user and a database so there’s something to connect to:
$ createdb -O bandit bandit
Then we register the Postgrex process as part of our main supervisor, so it starts alongside our webserver:
bandit_example/lib/application.ex
L#10:
children = [
– {Bandit, plug: BanditExample.MyPlug, port: 8080}
+ {Bandit, plug: BanditExample.MyPlug, port: 8080},
+ {Postgrex,
+ [
+ name: :bandit_db,
+ hostname: “localhost”,
+ username: “bandit”,
+ password: “bandit”,
+ database: “bandit”
+ ]
+ }
# Starts a worker by calling: BanditExample.Worker.start_link(arg)
And now we can jam a database query into our plug to prove it all works:
bandit_example/lib/my_plug.ex
L#6:
def call(conn, _options) do
+ %Postgrex.Result{rows: [[current_time]]} =
+ Postgrex.query(:bandit_db, “SELECT NOW() as current_time”, [])
+
conn
|> put_resp_content_type(“text/plain”)
– |> send_resp(200, “Hello, World!”)
+ |> send_resp(200, “Hello, World! It’s #{current_time}”)
end
And indeed it works:
anditExample.MyPlug with Band | Hello, World! It’s 2024-03-17
it 1.3.0 at 0.0.0.0:8080 (htt | 19:04:24.547992Z%
p) |
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] |
|
Interactive Elixir (1.16.1) – |
press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
🎉 We now have a basic web app connected to its database.
Granted, “basic” is doing a lot of heavy lifting in that sentence, because we would still have much work to do to have a truly useful and flexible app (For starters we could probably do better than jamming a raw query directly into a plug, to decouple HTML concerns from data-persistence concerns). But instead of getting stuck in that, let’s try out the next solution and see what it has to offer.
ℹ️ BTW the full code to this section is here.
Phoenix Framework
Of course we’re not skipping Phoenix in this article, it’s the elephant in the room for any topic that intersects Elixir and web. And it’s not unearned: Phoenix is a powerful and flexible framework, with tons of great developer-ergonomic features, and it supports creating all kinds of web applications small and huge.
ℹ️ BTW I’m usually weary of big web frameworks, having often gotten tangled up in their abstractions in other languages and come away regretting their hard-to-debug abstractions. Instead I prefer composing individual dependencies together. But… Phoenix is different, as we’ll see hints of in the below exploration.
Phoenix has a very easy way to get started, courtesy of a well-documented Getting Started guide that lay out the commands to generate and bootstrap an entire Phoenix sample app:
$ mix phx.new my_app # answer yes when prompted
$ cd my_app
$ mkdir -p priv/db && initdb -D priv/db && pg_ctl -D priv/db -l logfile start && createuser -d postgres
$ mix deps.get && mix ecto.create
ℹ️ BTW if you get errors from the database command you might already be running a database from earlier sections. You can reset Postgres using these commands as necessary, to get back on track:
$ lsof -ti :5432 | xargs -I {} kill {}; rm -rf priv/db # Kill all processes on Postgres’ default port
$ rm -rf priv/db # Delete the local DB data folder
And done!, this gets us a web-app we can start with iex -S mix phx.server and visit by browsing to http://localhost:4000:
What Phoenix Gives Us
The Phoenix generator made us a project that already connects to our Postgres database, which is quite convenient. It does this by integrating with Ecto, which is a highly popular Elixir database wrapper and query generator that itself uses Postgrex under the hood to connect to the database.
If we check out config/dev.exs we can actually spot the same pattern of hardcoded database credentials as we used in the Bandit section:
…
# Configure your database
config :my_app, MyApp.Repo,
username: “postgres”,
password: “postgres”,
hostname: “localhost”,
database: “my_app_dev”,
…
Well, almost like we did in Bandit section, because this configures something called MyApp.Repo. That’s a module we can see is implementing an Ecto.Repo behavior and it gets started as a child of the main supervisor very much like we did with Postgrex:
defmodule MyApp.Repo do
use Ecto.Repo,
otp_app: :my_app,
adapter: Ecto.Adapters.Postgres
end
$ cat lib/my_app/application.ex
…
def start(_type, _args) do
children = [
…
MyApp.Repo,
…
And that adapter: Ecto.Adapters.Postgres configuration field is how Ecto gets configured to resolve queries via Postgrex. So, when Ecto generates an SQL query we can imagine how it passes it off to Postgrex somewhere within its logic. Ecto even requires us to depend on Postgrex as a dependency to use that adapter, and indeed we can see Postgrex in our list of dependencies:
{:postgrex, “>= 0.0.0“},
To actually use the database we can create a module that does a query, and then call that function from our page controller:
my_app/lib/my_app/query.ex
L#1:
+defmodule MyApp.Query do
+ import Ecto.Query
+
+ alias MyApp.Repo
+
+ def get_db_time do
+ # The SELECT 1 is a dummy table to perform a query without a table
+ query = from u in fragment(“SELECT 1”), select: fragment(“NOW()”)
+ query |> Repo.all() |> List.first()
+ end
+end
my_app/lib/my_app_web/controllers/page_controller.ex
L#6:
# so skip the default app layout.
– render(conn, :home, layout: false)
+ db_time = MyApp.Query.get_db_time()
+ render(conn, :home, layout: false, db_time: db_time)
end
my_app/lib/my_app_web/controllers/page_html/home.html.heex
L#42:
<div class=”mx-auto max-w-xl lg:mx-0″>
+ <h1 class=”text-lg”>Database Time: <%= @db_time %></h1>
<svg viewBox=”0 0 71 48″ class=”h-12″ aria-hidden=”true”>
It’s not a visually stunning result, but we’re certainly making a database query 🎉:
ℹ️ BTW the full code to this section is here.
It’s the ability to follow these abstractions that makes me particularly happy with Phoenix: It’s not a framework with a hundred levels of abstracted and complex code; if we wanted to add proper database support to our own Bandit project we’d end up doing it pretty much exactly like Phoenix is doing it by pulling in Ecto and configuring the whole thing from a configuration file. It’s… simple. Which is a tremendous quality.
Conclusion
It’s been a lot of fun to try various solutions, but it’s hard to see a realistic alternative to Phoenix for our needs. I’m sure there are contexts where it makes sense to not choose Phoenix, but for our case it matches like a glove: Simple and easy to get started with, supporting incredible scale, with several really impressive features that we didn’t even touch on today, with clear and easy to approach documentation, and tons of examples in various articles to lean on. Phoenix is a juggernaut in the Elixir community and for good reason.
So I’ll continue this article-series based on Phoenix.
