Jekyll2022-09-29T20:36:00+00:00https://plippe.github.io/blog/feed.xmlPhilippe’s BlogPhilippe VinchonRust Wasm Github2021-07-12T00:00:00+00:002021-07-12T00:00:00+00:00https://plippe.github.io/blog/2021/07/12/rust-wasm-github<p>Based on <a href="https://www.arewewebyet.org/">Are we web yet?</a>, Rust is ready for web development. There are a lot of frameworks available. Most are to build backend services, but some are to develop frontends. That is right, you can write your next single-page application in Rust.</p>
<p>Bellow, I describe the steps needed to build, test, and deploy a Rust Wasm application on GitHub. When I say test and deploy, I do mean that GitHub will do the heavy lifting. It won’t just host the source code.</p>
<p>Let’s start with the Rust application.</p>
<h1 id="hello-world">Hello, World!</h1>
<p>I personally use <a href="https://yew.rs/">Yew</a>. This framework has good documentation, great examples, and is actively maintained. Their <a href="https://yew.rs/getting-started/build-a-sample-app">getting started</a> is also easy to follow. I simplified it, a tiny bit, as I want to focus on Yew in another post.</p>
<p>Before we start, make sure you have all the required tools installed. This refers to Rust and Cargo, of course, but also the following:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cargo <span class="nb">install </span>trunk wasm-bindgen-cli
rustup target add wasm32-unknown-unknown
</code></pre></div></div>
<p>With that out of the way, let’s create a Rust application.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cargo new rust-wasm-github
</code></pre></div></div>
<p>Then, add Yew as a dependency in <code class="language-plaintext highlighter-rouge">Cargo.toml</code>.</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">[dependencies]</span>
<span class="py">yew</span> <span class="p">=</span> <span class="s">"0.18"</span>
</code></pre></div></div>
<p>Next, replace <code class="language-plaintext highlighter-rouge">src/main.rs</code> with the following code.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">yew</span><span class="p">::</span><span class="nn">prelude</span><span class="p">::</span><span class="o">*</span><span class="p">;</span>
<span class="k">struct</span> <span class="nb">Index</span><span class="p">;</span>
<span class="k">impl</span> <span class="n">Component</span> <span class="k">for</span> <span class="nb">Index</span> <span class="p">{</span>
<span class="k">type</span> <span class="n">Message</span> <span class="o">=</span> <span class="p">();</span>
<span class="k">type</span> <span class="n">Properties</span> <span class="o">=</span> <span class="p">();</span>
<span class="k">fn</span> <span class="nf">create</span><span class="p">(</span><span class="mi">_</span><span class="p">:</span> <span class="nn">Self</span><span class="p">::</span><span class="n">Properties</span><span class="p">,</span> <span class="mi">_</span><span class="p">:</span> <span class="n">ComponentLink</span><span class="o"><</span><span class="n">Self</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">Self</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">update</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">,</span> <span class="mi">_</span><span class="p">:</span> <span class="nn">Self</span><span class="p">::</span><span class="n">Message</span><span class="p">)</span> <span class="k">-></span> <span class="n">ShouldRender</span> <span class="p">{</span>
<span class="k">false</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">change</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">,</span> <span class="mi">_</span><span class="p">:</span> <span class="nn">Self</span><span class="p">::</span><span class="n">Properties</span><span class="p">)</span> <span class="k">-></span> <span class="n">ShouldRender</span> <span class="p">{</span>
<span class="k">false</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">view</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Html</span> <span class="p">{</span>
<span class="nd">html!</span> <span class="p">{</span>
<span class="o"><</span><span class="n">div</span><span class="o">></span>
<span class="p">{</span> <span class="s">"Hello, World!"</span> <span class="p">}</span>
<span class="o"></</span><span class="n">div</span><span class="o">></span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nn">yew</span><span class="p">::</span><span class="nn">start_app</span><span class="p">::</span><span class="o"><</span><span class="nb">Index</span><span class="o">></span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And finally, create an <code class="language-plaintext highlighter-rouge">index.html</code> file at the root of the project.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><!DOCTYPE html></span>
<span class="nt"><html></span>
<span class="nt"><head></span>
<span class="nt"><meta</span> <span class="na">charset=</span><span class="s">"utf-8"</span> <span class="nt">/></span>
<span class="nt"><title></span>Hello, World!<span class="nt"></title></span>
<span class="nt"></head></span>
<span class="nt"></html></span>
</code></pre></div></div>
<p>All this should allow you to start your server and see the unavoidable hello world.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>trunk serve
</code></pre></div></div>
<p>With a working application, let’s move on to continuous integration.</p>
<h1 id="continuous-integration">Continuous Integration</h1>
<p>Rust is a safe language, but bugs can still sneak in. CI helps to catch those issues before they break anything. <a href="https://github.com/features/actions">GitHub Actions</a> can easily do that.</p>
<p>There are three simple things to check with Rust code. The first is obviously tests, the second is the format, and the third is code smells with Clippy. All those can run in parallel to speed up the process.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># in .github/workflows/continuous_integration.yml</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">Continuous integration</span>
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">push</span><span class="pi">,</span> <span class="nv">pull_request</span><span class="pi">]</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">test</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">cargo test --all</span>
<span class="na">format</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">cargo fmt --all -- --check</span>
<span class="na">clippy</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">cargo clippy --all -- -D warnings</span>
</code></pre></div></div>
<p>To wrap up CI, don’t forget to add <a href="https://docs.github.com/en/github/administering-a-repository/defining-the-mergeability-of-pull-requests/managing-a-branch-protection-rule">branch protection rules</a>. Those will force pull requests to pass the checks. They help keep the main branch clean.</p>
<p>With that out of the way, we can look at continuous deployment.</p>
<h1 id="continuous-deployment">Continuous Deployment</h1>
<p>Releasing updates can get in the way of actually writing code. Keeping the deployment simple helps, but automating it is even better. CD releases the latest commits straight to production, after CI of course.</p>
<p>Wasm applications are just a set of static files. There is no need for Docker or anything complex. <a href="https://pages.github.com/">GitHub Pages</a> and Actions is all we need. The former to host the files and the latter to deploy them. This is quite easy, especially with the awesome work done by <a href="https://github.com/actions-rs">actions-rs</a>, <a href="https://github.com/jetli">Jet Li</a>, and <a href="https://github.com/peaceiris">Shohei Ueda</a>.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># in .github/workflows/continuous_deployment.yml</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">Continuous deployment</span>
<span class="na">on</span><span class="pi">:</span>
<span class="na">workflow_run</span><span class="pi">:</span>
<span class="na">branches</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">main</span><span class="pi">]</span>
<span class="na">workflows</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">Continuous integration</span><span class="pi">]</span>
<span class="na">types</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">completed</span><span class="pi">]</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">release</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions-rs/toolchain@v1</span>
<span class="na">with</span><span class="pi">:</span>
<span class="na">toolchain</span><span class="pi">:</span> <span class="s">stable</span>
<span class="na">target</span><span class="pi">:</span> <span class="s">wasm32-unknown-unknown</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">jetli/trunk-action@v0.1.0</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">jetli/wasm-bindgen-action@v0.1.0</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">trunk build --release</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">peaceiris/actions-gh-pages@v3</span>
<span class="na">if</span><span class="pi">:</span> <span class="s">github.ref == 'refs/heads/main'</span>
<span class="na">with</span><span class="pi">:</span>
<span class="na">github_token</span><span class="pi">:</span> <span class="s">$</span>
<span class="na">publish_dir</span><span class="pi">:</span> <span class="s">./dist</span>
</code></pre></div></div>
<p>The last remaining step is to <a href="https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site">enable GitHub Pages</a>. That will make the application available on <code class="language-plaintext highlighter-rouge">[GITHUB_USER_NAME].github.io/[GITHUB_REPOSITORY_NAME]</code>.</p>
<hr />
<p>There you have it, these are all the steps required to build, test, and deploy a Rust Wasm application on GitHub. While I used Yew, you can build a similar pipeline with <a href="https://seed-rs.org/">Seed</a> or any other framework.</p>
<p>To finish, if I was too quick or if you don’t want to implement the pipeline yourself, have a look at the <a href="https://github.com/plippe/rust-wasm-github/">source code</a>.</p>Philippe VinchonBased on Are we web yet?, Rust is ready for web development. There are a lot of frameworks available. Most are to build backend services, but some are to develop frontends. That is right, you can write your next single-page application in Rust.Rust Extension Methods2021-06-09T00:00:00+00:002021-06-09T00:00:00+00:00https://plippe.github.io/blog/2021/06/09/rust-extension-methods<p>Rust’s standard library is small. It was designed to be. This is why some functionalities appear to be missing and some snippets feel long. Let’s see how extension methods can help.</p>
<h2 id="extension-methods">Extension methods</h2>
<p><a href="https://en.wikipedia.org/wiki/Extension_method">Wikipedia</a> defines an extension method as “a method added to an object after the original object was compiled”. In other words, it allows new methods to be implemented for existing structs and enums.</p>
<p>In Rust, it is done with a trait.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">core</span><span class="p">::</span><span class="nn">fmt</span><span class="p">::</span><span class="n">Debug</span><span class="p">;</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">DebugExt</span><span class="p">:</span> <span class="n">Debug</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">debug</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{:?}"</span><span class="p">,</span> <span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="p">:</span> <span class="n">Debug</span><span class="o">></span> <span class="n">DebugExt</span> <span class="k">for</span> <span class="n">A</span> <span class="p">{}</span>
</code></pre></div></div>
<p>This silly example adds a <code class="language-plaintext highlighter-rouge">debug()</code> method to all types that implement <code class="language-plaintext highlighter-rouge">Debug</code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">trait</span> <span class="n">PrintlnExt</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">println</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">PrintlnExt</span> <span class="k">for</span> <span class="o">&</span><span class="nb">str</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">println</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span> <span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Similarly, this one adds a <code class="language-plaintext highlighter-rouge">println()</code> method, but only for <code class="language-plaintext highlighter-rouge">&str</code>.</p>
<p>Both examples aren’t particularly useful, but they show the idea. Below are a few more that are actually helpful.</p>
<h2 id="into">Into</h2>
<p><a href="https://typelevel.org/cats/">Cats</a> is a Scala library <a href="https://plippe.github.io/blog/hashtags/cats.html">I covered a few times before</a>. Unrelated to the functional programming aspect, it contains a few general-purpose extension methods. Those shave a few characters, but, more importantly, allow to read the code from left to right.</p>
<p>The first, <a href="https://typelevel.org/cats/api/cats/syntax/OptionIdOps.html">OptionIdOps</a>, is to create <code class="language-plaintext highlighter-rouge">Option</code> values.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">trait</span> <span class="n">OptionIdExt</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_some</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">Option</span><span class="o"><&</span><span class="n">Self</span><span class="o">></span> <span class="p">{</span>
<span class="nn">Option</span><span class="p">::</span><span class="nf">Some</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">into_some</span><span class="p">(</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">Option</span><span class="o"><</span><span class="n">Self</span><span class="o">></span> <span class="k">where</span> <span class="n">Self</span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="nn">Option</span><span class="p">::</span><span class="nf">Some</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="n">OptionIdExt</span> <span class="k">for</span> <span class="n">A</span> <span class="p">{}</span>
<span class="k">let</span> <span class="mi">_</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">u8</span><span class="o">></span> <span class="o">=</span> <span class="mi">0</span><span class="nf">.into_some</span><span class="p">();</span>
</code></pre></div></div>
<p>The second, <a href="https://typelevel.org/cats/api/cats/syntax/EitherIdOps.html">EitherIdOps</a>, written for Scala’s <code class="language-plaintext highlighter-rouge">Either</code>, can be adapted for Rust’s <code class="language-plaintext highlighter-rouge">Result</code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">trait</span> <span class="n">ResultIdExt</span> <span class="p">{</span>
<span class="k">fn</span> <span class="n">as_ok</span><span class="o"><</span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><&</span><span class="n">Self</span><span class="p">,</span> <span class="n">E</span><span class="o">></span> <span class="p">{</span>
<span class="nn">Result</span><span class="p">::</span><span class="nf">Ok</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="n">into_ok</span><span class="o"><</span><span class="n">E</span><span class="o">></span><span class="p">(</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">Self</span><span class="p">,</span> <span class="n">E</span><span class="o">></span> <span class="k">where</span> <span class="n">Self</span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="nn">Result</span><span class="p">::</span><span class="nf">Ok</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="n">as_err</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="o">&</span><span class="n">Self</span><span class="o">></span> <span class="p">{</span>
<span class="nn">Result</span><span class="p">::</span><span class="nf">Err</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="n">into_err</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="n">Self</span><span class="o">></span> <span class="k">where</span> <span class="n">Self</span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="nn">Result</span><span class="p">::</span><span class="nf">Err</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="n">ResultIdExt</span> <span class="k">for</span> <span class="n">A</span> <span class="p">{}</span>
<span class="k">let</span> <span class="mi">_</span><span class="p">:</span> <span class="n">Result</span><span class="o"><</span><span class="nb">u8</span><span class="p">,</span> <span class="o">&</span><span class="nb">str</span><span class="o">></span> <span class="o">=</span> <span class="mi">0</span><span class="nf">.into_ok</span><span class="p">();</span>
</code></pre></div></div>
<p>All these methods don’t offer anything new. Their benefit comes from the extra readability. There is no longer any need to wrap a variable, a statement, or many statements in a constructor.</p>
<p>There is another solution to reduce nested statements.</p>
<h2 id="pipe">Pipe</h2>
<p>The <a href="https://en.wikipedia.org/wiki/Pipeline_(Unix)">pipe operator</a> is a way of listing statements instead of nesting them. The first statement runs … first. Its output is the input of the next statement. This repeats until all statements have run.</p>
<p>This concept can sound complicated, but it makes code much easier to read. For example, the following bash script calls <a href="https://store.steampowered.com/">Steam</a>’s API, sorts all characters, removes duplicates, and counts those that remain.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-s</span> https://api.steampowered.com/ISteamApps/GetAppList/v2 |
<span class="nb">grep</span> <span class="nt">-o</span> <span class="nb">.</span> |
<span class="nb">sort</span> |
<span class="nb">uniq</span> |
<span class="nb">wc</span> <span class="nt">-l</span>
</code></pre></div></div>
<p>While there is no value in this script, it shows the readability of the pipe operator.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">trait</span> <span class="n">PipeIdExt</span> <span class="p">{</span>
<span class="k">fn</span> <span class="n">pipe</span><span class="o"><</span><span class="n">A</span><span class="p">,</span> <span class="n">F</span><span class="o">></span><span class="p">(</span><span class="k">self</span><span class="p">,</span> <span class="n">f</span><span class="p">:</span> <span class="n">F</span><span class="p">)</span> <span class="k">-></span> <span class="n">A</span>
<span class="k">where</span>
<span class="n">Self</span><span class="p">:</span> <span class="n">Sized</span><span class="p">,</span>
<span class="n">F</span><span class="p">:</span> <span class="nf">FnOnce</span><span class="p">(</span><span class="n">Self</span><span class="p">)</span> <span class="k">-></span> <span class="n">A</span><span class="p">,</span>
<span class="p">{</span>
<span class="nf">f</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="n">PipeIdExt</span> <span class="k">for</span> <span class="n">A</span> <span class="p">{}</span>
<span class="s">"Hello, world!"</span><span class="nf">.pipe</span><span class="p">(|</span><span class="n">s</span><span class="p">|</span> <span class="nd">println!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span> <span class="n">s</span><span class="p">));</span>
</code></pre></div></div>
<p>Those few examples should allow you to implement your own extension methods.</p>
<hr />
<p>As a closing note, I would like to add few words of caution. Extension methods have drawbacks. While they improve readability, they impact onboarding and maintainability. Beware of writing a DSL that needs to be taught and spaghetti code that increase coupling.</p>
<p>I found extension methods work best when they link an existing object to an existing method, but to each their own.</p>Philippe VinchonRust’s standard library is small. It was designed to be. This is why some functionalities appear to be missing and some snippets feel long. Let’s see how extension methods can help.Playing With Scala - Slick2020-06-01T00:00:00+00:002020-06-01T00:00:00+00:00https://plippe.github.io/blog/2020/06/01/playing-with-scala-slick<p>Applications that interact with databases have a few moving pieces to keep in mind. Between the connection, the setup, and the queries, there is enough to find it overwhelming. This post will cover all those pieces in a Play application.</p>
<p>To avoid repeating myself, I will start with a working CRUD application. A simple ReST API that uses a <code class="language-plaintext highlighter-rouge">Map</code> instead of a database. It is like the <a href="/blog/2020/04/01/playing-with-scala-rest-api.html">Pet Store we built</a>, but for recipes.</p>
<p>If at any point you want to jump ahead, the application is available on <a href="https://github.com/plippe/playing-with-scala-slick">GitHub</a>. It has a few improvements, but nothing major.</p>
<p>Before starting, we need a PostgreSQL server running. If you do not wish to install one, Docker is a great alternative.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="se">\</span>
<span class="nt">--env</span> <span class="nv">POSTGRES_DB</span><span class="o">=</span>db <span class="se">\</span>
<span class="nt">--env</span> <span class="nv">POSTGRES_USER</span><span class="o">=</span>user <span class="se">\</span>
<span class="nt">--env</span> <span class="nv">POSTGRES_PASSWORD</span><span class="o">=</span>password <span class="se">\</span>
<span class="nt">--publish</span> 5432:5432 <span class="se">\</span>
<span class="nt">--rm</span> <span class="se">\</span>
<span class="nt">--interactive</span> <span class="se">\</span>
<span class="nt">--tty</span> <span class="se">\</span>
postgres
</code></pre></div></div>
<p>With our recipes API and a database running, let’s jump right in.</p>
<h2 id="data-access-object">Data Access Object</h2>
<p>The controller currently holds a <code class="language-plaintext highlighter-rouge">Map</code> to store recipes.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/RecipesController.scala</span>
<span class="c1">// In RecipesController class</span>
<span class="k">val</span> <span class="nv">store</span> <span class="k">=</span> <span class="nv">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">UUID</span>, <span class="kt">models.Recipe</span><span class="o">]</span>
</code></pre></div></div>
<p>This is an easy way to cut corners for a proof of concept but it isn’t good code. One issue is around the separation of concerns. The controller and the data access object should be in distinct classes. Injecting a DAO also offers more flexibility like swapping one for another.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">package</span> <span class="nn">daos</span>
<span class="k">import</span> <span class="nn">java.util.UUID</span>
<span class="k">trait</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Unit</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Unit</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Unit</span>
<span class="o">}</span>
</code></pre></div></div>
<p>For a simple transition, <code class="language-plaintext highlighter-rouge">Map</code> needs an implementation.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">import</span> <span class="nn">javax.inject.Singleton</span>
<span class="nd">@Singleton</span>
<span class="k">class</span> <span class="nc">RecipesDaoMap</span> <span class="k">extends</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">map</span> <span class="k">=</span> <span class="nv">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">UUID</span>, <span class="kt">models.Recipe</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">map</span><span class="o">.</span><span class="py">values</span><span class="o">.</span><span class="py">toSeq</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">map</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">map</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">recipe</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">map</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">recipe</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">map</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This DAO requires <code class="language-plaintext highlighter-rouge">@Singleton</code>. It guarantees that only a single instance can exist and thus a single <code class="language-plaintext highlighter-rouge">Map</code>. It is like Scala’s <code class="language-plaintext highlighter-rouge">object</code> but for Java’s dependency injection.</p>
<p>Adding the <code class="language-plaintext highlighter-rouge">RecipesDao</code> to the <code class="language-plaintext highlighter-rouge">RecipesController</code> allows the <code class="language-plaintext highlighter-rouge">dao</code> variable to replace the <code class="language-plaintext highlighter-rouge">store</code> one.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/RecipesController.scala</span>
<span class="k">class</span> <span class="nc">RecipesController</span> <span class="nd">@Inject</span><span class="o">()(</span>
<span class="n">dao</span><span class="k">:</span> <span class="kt">daos.RecipesDao</span><span class="o">,</span>
<span class="k">val</span> <span class="nv">controllerComponents</span><span class="k">:</span> <span class="kt">ControllerComponents</span>
<span class="o">)(</span><span class="k">implicit</span> <span class="n">ec</span><span class="k">:</span> <span class="kt">ExecutionContext</span><span class="o">)</span> <span class="k">extends</span> <span class="nc">BaseController</span> <span class="o">{</span>
<span class="o">...</span>
<span class="o">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">RecipesDao</code> is a trait. The desired implementation must be specified. <a href="https://github.com/google/guice">Guice</a> offers <code class="language-plaintext highlighter-rouge">@ImplementedBy</code> as a solution.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">import</span> <span class="nn">com.google.inject.ImplementedBy</span>
<span class="nd">@ImplementedBy</span><span class="o">[</span><span class="kt">RecipesDaoMap</span><span class="o">]</span>
<span class="k">trait</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="o">...</span>
<span class="o">}</span>
</code></pre></div></div>
<p>I prefer Play’s <code class="language-plaintext highlighter-rouge">Module</code>. It removes the explicit dependency on Guice. It allows to define all bindings together. Furthermore, it can bind implementations to traits from different libraries.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/Module.scala</span>
<span class="k">import</span> <span class="nn">com.google.inject.AbstractModule</span>
<span class="k">class</span> <span class="nc">Module</span> <span class="k">extends</span> <span class="nc">AbstractModule</span> <span class="o">{</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">configure</span><span class="o">()</span> <span class="k">=</span>
<span class="nf">bind</span><span class="o">(</span><span class="n">classOf</span><span class="o">[</span><span class="kt">daos.RecipesDao</span><span class="o">]).</span><span class="py">to</span><span class="o">(</span><span class="n">classOf</span><span class="o">[</span><span class="kt">daos.RecipesDaoMap</span><span class="o">])</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This is all quite simple, but Slick can’t conform to this interface. It returns <code class="language-plaintext highlighter-rouge">Future</code>s.</p>
<h2 id="future">Future</h2>
<p><code class="language-plaintext highlighter-rouge">Future</code>s are a way of running code asynchronously. A job is given to an execution context. The result is available when the job completes. Instead of waiting patiently, the application keeps working on other available jobs.</p>
<p>With Slick returning <code class="language-plaintext highlighter-rouge">Future</code>s, the <code class="language-plaintext highlighter-rouge">RecipesDao</code> and its implementations must return them too.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">import</span> <span class="nn">scala.concurrent.Future</span>
<span class="k">trait</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span>
<span class="o">}</span>
<span class="nd">@Singleton</span>
<span class="k">class</span> <span class="nc">RecipesDaoMap</span> <span class="k">extends</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">map</span> <span class="k">=</span> <span class="nv">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">UUID</span>, <span class="kt">models.Recipe</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">Future</span><span class="o">.</span><span class="py">successful</span><span class="o">(</span><span class="nv">map</span><span class="o">.</span><span class="py">values</span><span class="o">.</span><span class="py">toSeq</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">Future</span><span class="o">.</span><span class="py">successful</span><span class="o">(</span><span class="nv">map</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">Future</span><span class="o">.</span><span class="py">successful</span><span class="o">(</span><span class="nv">map</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">recipe</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">Future</span><span class="o">.</span><span class="py">successful</span><span class="o">(</span><span class="nv">map</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">recipe</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">Future</span><span class="o">.</span><span class="py">successful</span><span class="o">(</span><span class="nv">map</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>These changes affect the <code class="language-plaintext highlighter-rouge">RecipesController</code>.</p>
<ul>
<li>Replace <code class="language-plaintext highlighter-rouge">Action</code>s by <code class="language-plaintext highlighter-rouge">Action.async</code>s</li>
<li>Inject and <code class="language-plaintext highlighter-rouge">implicit</code> <code class="language-plaintext highlighter-rouge">ExecutionContext</code></li>
<li>Use <code class="language-plaintext highlighter-rouge">.map</code> to alter a <code class="language-plaintext highlighter-rouge">Future</code>’s result</li>
<li>Use <code class="language-plaintext highlighter-rouge">.flatMap</code> when chaining <code class="language-plaintext highlighter-rouge">Future</code>s</li>
<li>…</li>
</ul>
<p>The new controller can be found on <a href="https://github.com/plippe/playing-with-scala-slick/blob/master/app/controllers/RecipesController.scala">GitHub</a>.</p>
<p>We could creating a Slick <code class="language-plaintext highlighter-rouge">RecipesDao</code>, but let’s create the SQL table first.</p>
<h2 id="evolutions">Evolutions</h2>
<p>Play uses <a href="https://www.playframework.com/documentation/2.8.x/Evolutions">Evolution</a> to apply database migrations. This is a great way of synchronizing database updates. It also gives a way of rolling back bad changes.</p>
<p>Add the library, and the relevant dependencies, to the <code class="language-plaintext highlighter-rouge">build.sbt</code> file.</p>
<pre><code class="language-sbt">// In /build.sbt
libraryDependencies ++= Seq(
"org.postgresql" % "postgresql" % "42.2.12",
"com.typesafe.play" %% "play-slick" % "5.0.0",
"com.typesafe.play" %% "play-slick-evolutions" % "5.0.0",
)
</code></pre>
<p>These allow Play to communicate with PostgreSQL, but it needs to connect first. The configuration file, <code class="language-plaintext highlighter-rouge">/conf/application.conf</code> holds the necessary settings.</p>
<div class="language-hocon highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In /conf/application.conf</span><span class="w">
</span><span class="nl">slick.dbs.default.profile</span><span class="p">=</span><span class="s2">"slick.jdbc.PostgresProfile$"</span><span class="w">
</span><span class="nl">slick.dbs.default.db.url</span><span class="p">=</span><span class="s2">"jdbc:postgresql://localhost:5432/db"</span><span class="w">
</span><span class="nl">slick.dbs.default.db.username</span><span class="p">=</span><span class="s2">"user"</span><span class="w">
</span><span class="nl">slick.dbs.default.db.password</span><span class="p">=</span><span class="s2">"password"</span><span class="w">
</span></code></pre></div></div>
<p>The migration scripts to apply are found in the <code class="language-plaintext highlighter-rouge">/conf/evolutions/default/</code> folder. The first file is <code class="language-plaintext highlighter-rouge">1.sql</code>, the second is <code class="language-plaintext highlighter-rouge">2.sql</code>, and so on.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- In /conf/evolutions/default/1.sql</span>
<span class="c1">-- !Ups</span>
<span class="k">create</span> <span class="k">table</span> <span class="n">recipes</span> <span class="p">(</span>
<span class="n">id</span> <span class="n">uuid</span> <span class="k">primary</span> <span class="k">key</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span> <span class="k">not</span> <span class="k">null</span><span class="p">,</span>
<span class="n">description</span> <span class="nb">text</span> <span class="k">not</span> <span class="k">null</span><span class="p">,</span>
<span class="n">created_at</span> <span class="nb">timestamp</span> <span class="k">not</span> <span class="k">null</span><span class="p">,</span>
<span class="n">updated_at</span> <span class="nb">timestamp</span> <span class="k">not</span> <span class="k">null</span>
<span class="p">);</span>
<span class="c1">-- !Downs</span>
<span class="k">drop</span> <span class="k">table</span> <span class="n">recipes</span><span class="p">;</span>
</code></pre></div></div>
<p>When Play starts, it checks that all migration scripts have been applied.</p>
<p><img src="https://plippe.github.io/blog/assets/images/posts/playing-with-scala-slick-evolution.png" alt="Source configuration" /></p>
<p>To avoid having to click a button, Slick can apply migrations automatically.</p>
<div class="language-hocon highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In /conf/application.conf</span><span class="w">
</span><span class="nl">play.evolutions.autoApply</span><span class="p">=</span><span class="kc">true</span><span class="w">
</span></code></pre></div></div>
<p>After starting a database management systems, connecting to it, and setting it up, we can finally create a <code class="language-plaintext highlighter-rouge">RecipesDaoSlick</code>.</p>
<h2 id="slick">Slick</h2>
<p>An empty <code class="language-plaintext highlighter-rouge">RecipesDaoSlick</code> only needs to extend <code class="language-plaintext highlighter-rouge">RecipesDao</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">class</span> <span class="nc">RecipesDaoSlick</span> <span class="k">extends</span> <span class="nc">RecipesDao</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="o">???</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="o">???</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="o">???</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="o">???</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="o">???</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Slick is compatible with many database management system. The differences are available via a profile. This project uses PostgreSQL but there is no need to hard code the <code class="language-plaintext highlighter-rouge">slick.jdbc.PostgresProfile</code>. Slick can use the <code class="language-plaintext highlighter-rouge">/conf/application.conf</code> to inject the appropriete one.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">import</span> <span class="nn">play.api.db.slick._</span>
<span class="k">import</span> <span class="nn">slick.jdbc.JdbcProfile</span>
<span class="k">import</span> <span class="nn">javax.inject.Inject</span>
<span class="k">class</span> <span class="nc">RecipesDaoSlick</span> <span class="nd">@Inject</span><span class="o">()(</span><span class="k">protected</span> <span class="k">val</span> <span class="nv">dbConfigProvider</span><span class="k">:</span> <span class="kt">DatabaseConfigProvider</span><span class="o">)</span>
<span class="k">extends</span> <span class="nc">RecipesDao</span>
<span class="k">with</span> <span class="nc">HasDatabaseConfigProvider</span><span class="o">[</span><span class="kt">JdbcProfile</span><span class="o">]</span> <span class="o">{</span>
<span class="k">import</span> <span class="nn">profile.api._</span>
<span class="o">...</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Slick is unable to cast a Scala case class to SQL table by itself. The mapping must be explicit.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="c1">// In RecipesDaoSlick class</span>
<span class="k">private</span> <span class="k">class</span> <span class="nc">RecipesTable</span><span class="o">(</span><span class="n">tag</span><span class="k">:</span> <span class="kt">Tag</span><span class="o">)</span> <span class="k">extends</span> <span class="nc">Table</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">](</span><span class="n">tag</span><span class="o">,</span> <span class="s">"recipes"</span><span class="o">)</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">id</span> <span class="k">=</span> <span class="n">column</span><span class="o">[</span><span class="kt">UUID</span><span class="o">](</span><span class="s">"id"</span><span class="o">,</span> <span class="nv">O</span><span class="o">.</span><span class="py">PrimaryKey</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">name</span> <span class="k">=</span> <span class="n">column</span><span class="o">[</span><span class="kt">String</span><span class="o">](</span><span class="s">"name"</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">description</span> <span class="k">=</span> <span class="n">column</span><span class="o">[</span><span class="kt">String</span><span class="o">](</span><span class="s">"description"</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">createdAt</span> <span class="k">=</span> <span class="n">column</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">](</span><span class="s">"created_at"</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">updatedAt</span> <span class="k">=</span> <span class="n">column</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">](</span><span class="s">"updated_at"</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">*</span> <span class="k">=</span> <span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="n">name</span><span class="o">,</span> <span class="n">description</span><span class="o">,</span> <span class="n">createdAt</span><span class="o">,</span> <span class="n">updatedAt</span><span class="o">).</span><span class="py">mapTo</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The code above requires a <code class="language-plaintext highlighter-rouge">tupled</code> method on the <code class="language-plaintext highlighter-rouge">Recipe</code> object. It is like apply, but takes all arguments as a tuple.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/Recipe.scala</span>
<span class="c1">// In Recipe object</span>
<span class="k">def</span> <span class="nf">tupled</span><span class="o">(</span><span class="n">tuple</span><span class="k">:</span> <span class="o">((</span><span class="kt">UUID</span><span class="o">,</span> <span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">,</span> <span class="nc">LocalDateTime</span><span class="o">,</span> <span class="nc">LocalDateTime</span><span class="o">)))</span><span class="k">:</span> <span class="kt">Recipe</span> <span class="o">=</span>
<span class="o">(</span><span class="n">apply</span> <span class="k">_</span><span class="o">).</span><span class="py">tupled</span><span class="o">(</span><span class="n">tuple</span><span class="o">)</span>
</code></pre></div></div>
<p>Slick has a query builder to avoid writing SQL. It uses <code class="language-plaintext highlighter-rouge">+=</code> for inserts, <code class="language-plaintext highlighter-rouge">filter</code> for where clauses, and much more.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="c1">// In RecipesDaoSlick class</span>
<span class="k">private</span> <span class="k">val</span> <span class="nv">table</span> <span class="k">=</span> <span class="nc">TableQuery</span><span class="o">[</span><span class="kt">RecipesTable</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="nv">table</span><span class="o">.</span><span class="py">result</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="nv">table</span><span class="o">.</span><span class="py">filter</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">id</span> <span class="o">===</span> <span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">result</span>
<span class="o">.</span><span class="py">headOption</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="o">(</span><span class="n">table</span> <span class="o">+=</span> <span class="n">recipe</span><span class="o">)</span>
<span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span> <span class="c1">// Return Unit instead of Int</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="nv">table</span><span class="o">.</span><span class="py">filter</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">id</span> <span class="o">===</span> <span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="n">recipe</span><span class="o">)</span>
<span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span> <span class="c1">// Return Unit instead of Int</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="nv">table</span><span class="o">.</span><span class="py">filter</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">id</span> <span class="o">===</span> <span class="nv">recipe</span><span class="o">.</span><span class="py">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">delete</span>
<span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span> <span class="c1">// Return Unit instead of Int</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Slick’s query builder is a controversial way to write safe SQL, but there is an alternative. We can write queries directly.</p>
<h2 id="slick-plain-sql">Slick Plain SQL</h2>
<p>Slick has many string interpolations methods to write SQL. Two are described below.</p>
<p>The first is <code class="language-plaintext highlighter-rouge">sql</code>. It returns a result set. The <code class="language-plaintext highlighter-rouge">.as</code> method casts the record into a specific Scala type.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="n">sql</span><span class="s">"""
select id, name, description, created_at, updated_at
from recipes
"""</span><span class="o">.</span><span class="py">as</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The query can also contain a where clause with variables. The string interpolation will protect against <a href="https://en.wikipedia.org/wiki/SQL_injection">SQL injections</a>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="n">sql</span><span class="s">"""
select id, name, description, created_at, updated_at
from recipes
where id = ${id}
"""</span><span class="o">.</span><span class="py">as</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">].</span><span class="py">headOption</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The second string interpolation is <code class="language-plaintext highlighter-rouge">sqlu</code>. It returns the number of records affected by the query.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span> <span class="o">{</span>
<span class="n">sqlu</span><span class="s">"""
insert into recipes(id, name, description, created_at, updated_at)
values (${recipe.id}, ${recipe.name}, ${recipe.description}, ${recipe.createdAt}, ${recipe.updatedAt})
"""</span><span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Slick’s uses implicit <code class="language-plaintext highlighter-rouge">SetParameter</code>s to build queries and <code class="language-plaintext highlighter-rouge">GetResult</code>s to parse records. Many types are plug and play, but <code class="language-plaintext highlighter-rouge">UUID</code>, <code class="language-plaintext highlighter-rouge">LocalDateTime</code>, and <code class="language-plaintext highlighter-rouge">Recipe</code> aren’t.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nn">java.sql.</span><span class="o">{</span><span class="nc">Timestamp</span><span class="o">,</span> <span class="nc">Types</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">slick.jdbc.</span><span class="o">{</span><span class="nc">GetResult</span><span class="o">,</span> <span class="nc">SetParameter</span><span class="o">}</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickGetResultLocalDateTime</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">r</span><span class="o">.</span><span class="py">nextTimestamp</span><span class="o">.</span><span class="py">toLocalDateTime</span><span class="o">)</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickSetParameterLocalDateTime</span><span class="k">:</span> <span class="kt">SetParameter</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">SetParameter</span> <span class="o">{</span> <span class="nf">case</span> <span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="n">pp</span><span class="o">)</span> <span class="k">=></span> <span class="nv">pp</span><span class="o">.</span><span class="py">setTimestamp</span><span class="o">(</span><span class="nv">Timestamp</span><span class="o">.</span><span class="py">valueOf</span><span class="o">(</span><span class="n">v</span><span class="o">))</span> <span class="o">}</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickGetResultUUID</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">UUID</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">r</span><span class="o">.</span><span class="py">nextObject</span><span class="o">.</span><span class="py">asInstanceOf</span><span class="o">[</span><span class="kt">UUID</span><span class="o">])</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickSetParameterUUID</span><span class="k">:</span> <span class="kt">SetParameter</span><span class="o">[</span><span class="kt">UUID</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">SetParameter</span> <span class="o">{</span> <span class="nf">case</span> <span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="n">pp</span><span class="o">)</span> <span class="k">=></span> <span class="nv">pp</span><span class="o">.</span><span class="py">setObject</span><span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="nv">Types</span><span class="o">.</span><span class="py">OTHER</span><span class="o">)</span> <span class="o">}</span>
<span class="k">implicit</span> <span class="k">val</span> <span class="nv">getRecipeResult</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span> <span class="k">=</span> <span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">models</span><span class="o">.</span><span class="py">Recipe</span><span class="o">(</span><span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<))</span>
</code></pre></div></div>
<p>Composing all those parts make a <code class="language-plaintext highlighter-rouge">RecipesDaoSlickPlainSql</code> data access object.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/daos/RecipesDao.scala</span>
<span class="k">import</span> <span class="nn">java.sql.</span><span class="o">{</span><span class="nc">Timestamp</span><span class="o">,</span> <span class="nc">Types</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">slick.jdbc.</span><span class="o">{</span><span class="nc">GetResult</span><span class="o">,</span> <span class="nc">SetParameter</span><span class="o">}</span>
<span class="k">object</span> <span class="nc">jdbc</span> <span class="o">{</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickGetResultLocalDateTime</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">r</span><span class="o">.</span><span class="py">nextTimestamp</span><span class="o">.</span><span class="py">toLocalDateTime</span><span class="o">)</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickSetParameterLocalDateTime</span><span class="k">:</span> <span class="kt">SetParameter</span><span class="o">[</span><span class="kt">LocalDateTime</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">SetParameter</span> <span class="o">{</span> <span class="nf">case</span> <span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="n">pp</span><span class="o">)</span> <span class="k">=></span> <span class="nv">pp</span><span class="o">.</span><span class="py">setTimestamp</span><span class="o">(</span><span class="nv">Timestamp</span><span class="o">.</span><span class="py">valueOf</span><span class="o">(</span><span class="n">v</span><span class="o">))</span> <span class="o">}</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickGetResultUUID</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">UUID</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">r</span><span class="o">.</span><span class="py">nextObject</span><span class="o">.</span><span class="py">asInstanceOf</span><span class="o">[</span><span class="kt">UUID</span><span class="o">])</span>
<span class="k">implicit</span> <span class="k">final</span> <span class="k">def</span> <span class="nf">helpersSlickSetParameterUUID</span><span class="k">:</span> <span class="kt">SetParameter</span><span class="o">[</span><span class="kt">UUID</span><span class="o">]</span> <span class="k">=</span>
<span class="nc">SetParameter</span> <span class="o">{</span> <span class="nf">case</span> <span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="n">pp</span><span class="o">)</span> <span class="k">=></span> <span class="nv">pp</span><span class="o">.</span><span class="py">setObject</span><span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="nv">Types</span><span class="o">.</span><span class="py">OTHER</span><span class="o">)</span> <span class="o">}</span>
<span class="o">}</span>
<span class="k">class</span> <span class="nc">RecipesDaoSlickPlainSql</span> <span class="nd">@Inject</span><span class="o">()(</span><span class="k">protected</span> <span class="k">val</span> <span class="nv">dbConfigProvider</span><span class="k">:</span> <span class="kt">DatabaseConfigProvider</span><span class="o">)</span> <span class="k">extends</span> <span class="nc">RecipesDao</span> <span class="k">with</span> <span class="nc">HasDatabaseConfigProvider</span><span class="o">[</span><span class="kt">JdbcProfile</span><span class="o">]</span> <span class="o">{</span>
<span class="k">import</span> <span class="nn">profile.api._</span>
<span class="k">import</span> <span class="nn">jdbc._</span>
<span class="k">implicit</span> <span class="k">val</span> <span class="nv">getRecipeResult</span><span class="k">:</span> <span class="kt">GetResult</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]</span> <span class="k">=</span> <span class="nc">GetResult</span><span class="o">(</span><span class="n">r</span> <span class="k">=></span> <span class="nv">models</span><span class="o">.</span><span class="py">Recipe</span><span class="o">(</span><span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<,</span> <span class="n">r</span><span class="o">.<<))</span>
<span class="k">def</span> <span class="nf">findAll</span><span class="o">()</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Seq</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="n">sql</span><span class="s">"""
select id, name, description, created_at, updated_at
from recipes
"""</span><span class="o">.</span><span class="py">as</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">])</span>
<span class="k">def</span> <span class="nf">findById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Option</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">]]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="n">sql</span><span class="s">"""
select id, name, description, created_at, updated_at
from recipes
where id = ${id}
"""</span><span class="o">.</span><span class="py">as</span><span class="o">[</span><span class="kt">models.Recipe</span><span class="o">].</span><span class="py">headOption</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">insert</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="n">sqlu</span><span class="s">"""
insert into recipes(id, name, description, created_at, updated_at)
values (${recipe.id}, ${recipe.name}, ${recipe.description}, ${recipe.createdAt}, ${recipe.updatedAt})
"""</span><span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span>
<span class="k">def</span> <span class="nf">update</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="n">sqlu</span><span class="s">"""
update recipes
set name = ${recipe.name},
description = ${recipe.description},
created_at = ${recipe.createdAt},
updated_at = ${recipe.updatedAt}
where id = ${recipe.id}
"""</span><span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span>
<span class="k">def</span> <span class="nf">delete</span><span class="o">(</span><span class="n">recipe</span><span class="k">:</span> <span class="kt">models.Recipe</span><span class="o">)</span><span class="k">:</span> <span class="kt">Future</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span> <span class="nv">db</span><span class="o">.</span><span class="py">run</span><span class="o">(</span><span class="n">sqlu</span><span class="s">"""
delete from recipes
where id = ${recipe.id}
"""</span><span class="o">.</span><span class="py">andThen</span><span class="o">(</span><span class="nv">DBIOAction</span><span class="o">.</span><span class="py">successful</span><span class="o">(()))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The application should now have three different <code class="language-plaintext highlighter-rouge">RecipesDao</code>s. Two of those read from and write to a PostgreSQL database. You just need to bind the one you prefer in the <code class="language-plaintext highlighter-rouge">/app/Module.scala</code>.</p>
<hr />
<p>That was a lot to cover: Dependency Injection, Future, Evolution, Slick, Slick Plain SQL. I could have definitely broken this into many parts, but we got here in the end.</p>
<p>We have a Play application that setups and queries a database, but does it work. Next time, we will make sure it behaves correctly by writing tests.</p>Philippe VinchonApplications that interact with databases have a few moving pieces to keep in mind. Between the connection, the setup, and the queries, there is enough to find it overwhelming. This post will cover all those pieces in a Play application.Playing With Scala - ReST UI2020-05-01T00:00:00+00:002020-05-01T00:00:00+00:00https://plippe.github.io/blog/2020/05/01/playing-with-scala-rest-ui<p>Building a single page application is a luxury not everyone can afford. If you are short on time, don’t learn <a href="https://vuejs.org/">Vue.js</a>, <a href="https://reactjs.org/">React</a>, or the other flavor of the month. Use Play Framework’s built-in templating engine, <a href="https://www.playframework.com/documentation/2.8.x/ScalaTemplates">Twirl</a>. It isn’t perfect but gets the job done.</p>
<p>This post will help you build a blog application like the <a href="https://guides.rubyonrails.org/getting_started.html#getting-up-and-running">Ruby on Rails Guide - Getting Up and Running</a>. If you only care about the code, you can head straight to <a href="https://github.com/plippe/playing-with-scala-rest-ui">GitHub</a> for the working application.</p>
<p>Let’s jump right in with the models.</p>
<h2 id="models">Models</h2>
<p>A blog application needs to represent articles. We aren’t building the next <a href="https://medium.com/">Medium</a>, a handful of attributes should do the trick.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/Article.scala</span>
<span class="k">package</span> <span class="nn">models</span>
<span class="k">import</span> <span class="nn">java.time.LocalDateTime</span>
<span class="k">import</span> <span class="nn">java.util.UUID</span>
<span class="k">case</span> <span class="k">class</span> <span class="nc">Article</span><span class="o">(</span>
<span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">,</span>
<span class="n">title</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">text</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">createdAt</span><span class="k">:</span> <span class="kt">LocalDateTime</span><span class="o">,</span>
<span class="n">updatedAt</span><span class="k">:</span> <span class="kt">LocalDateTime</span><span class="o">,</span>
<span class="o">)</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Option</code>s are an easy way to ignore fields that users shouldn’t submit. An alternative is to create a completely different class.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/ArticleForm.scala</span>
<span class="k">package</span> <span class="nn">models</span>
<span class="k">case</span> <span class="k">class</span> <span class="nc">ArticleForm</span><span class="o">(</span>
<span class="n">title</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">text</span><span class="k">:</span> <span class="kt">String</span>
<span class="o">)</span>
</code></pre></div></div>
<p>To centralize the conversion logic closer to the classes, we can add a few factory methods.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/Article.scala</span>
<span class="k">object</span> <span class="nc">Article</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">fromForm</span><span class="o">(</span><span class="n">form</span><span class="k">:</span> <span class="kt">ArticleForm</span><span class="o">)</span><span class="k">:</span> <span class="kt">Article</span> <span class="o">=</span>
<span class="nc">Article</span><span class="o">(</span>
<span class="n">id</span> <span class="k">=</span> <span class="nv">UUID</span><span class="o">.</span><span class="py">randomUUID</span><span class="o">,</span>
<span class="n">title</span> <span class="k">=</span> <span class="nv">form</span><span class="o">.</span><span class="py">title</span><span class="o">,</span>
<span class="n">text</span> <span class="k">=</span> <span class="nv">form</span><span class="o">.</span><span class="py">text</span><span class="o">,</span>
<span class="n">createdAt</span> <span class="k">=</span> <span class="nv">LocalDateTime</span><span class="o">.</span><span class="py">now</span><span class="o">(),</span>
<span class="n">updatedAt</span> <span class="k">=</span> <span class="nv">LocalDateTime</span><span class="o">.</span><span class="py">now</span><span class="o">(),</span>
<span class="o">)</span>
<span class="k">def</span> <span class="nf">updated</span><span class="o">(</span><span class="n">self</span><span class="k">:</span> <span class="kt">Article</span><span class="o">)(</span><span class="n">form</span><span class="k">:</span> <span class="kt">ArticleForm</span><span class="o">)</span><span class="k">:</span> <span class="kt">Article</span> <span class="o">=</span>
<span class="nv">self</span><span class="o">.</span><span class="py">copy</span><span class="o">(</span>
<span class="n">title</span> <span class="k">=</span> <span class="nv">form</span><span class="o">.</span><span class="py">title</span><span class="o">,</span>
<span class="n">text</span> <span class="k">=</span> <span class="nv">form</span><span class="o">.</span><span class="py">text</span><span class="o">,</span>
<span class="n">updatedAt</span> <span class="k">=</span> <span class="nv">LocalDateTime</span><span class="o">.</span><span class="py">now</span><span class="o">(),</span>
<span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/ArticleForm.scala</span>
<span class="k">object</span> <span class="nc">ArticleForm</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">fromModel</span><span class="o">(</span><span class="n">model</span><span class="k">:</span> <span class="kt">Article</span><span class="o">)</span><span class="k">:</span> <span class="kt">ArticleForm</span> <span class="o">=</span>
<span class="nc">ArticleForm</span><span class="o">(</span>
<span class="n">title</span> <span class="k">=</span> <span class="nv">model</span><span class="o">.</span><span class="py">title</span><span class="o">,</span>
<span class="n">text</span> <span class="k">=</span> <span class="nv">model</span><span class="o">.</span><span class="py">text</span><span class="o">,</span>
<span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This is all great for us, but we need to inform Play about those types.</p>
<h2 id="form">Form</h2>
<p>Play uses the <a href="https://www.playframework.com/documentation/2.8.x/api/scala/play/api/data/Form.html"><code class="language-plaintext highlighter-rouge">Form</code></a> class to handle form submissions. At its core, it attempts to convert a request to a specific type. This is perfect to build an <code class="language-plaintext highlighter-rouge">ArticleForm</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/ArticleForm.scala</span>
<span class="k">import</span> <span class="nn">play.api.data.Form</span>
<span class="k">import</span> <span class="nn">play.api.data.Forms.</span><span class="o">{</span><span class="n">mapping</span><span class="o">,</span> <span class="n">of</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">play.api.data.format.Formats._</span>
<span class="c1">// In ArticleForm object</span>
<span class="k">val</span> <span class="nv">playForm</span><span class="k">:</span> <span class="kt">Form</span><span class="o">[</span><span class="kt">ArticleForm</span><span class="o">]</span> <span class="k">=</span> <span class="nc">Form</span><span class="o">(</span><span class="nf">mapping</span><span class="o">(</span>
<span class="s">"title"</span> <span class="o">-></span> <span class="n">of</span><span class="o">[</span><span class="kt">String</span><span class="o">],</span>
<span class="s">"text"</span> <span class="o">-></span> <span class="n">of</span><span class="o">[</span><span class="kt">String</span><span class="o">],</span>
<span class="o">)(</span><span class="nv">ArticleForm</span><span class="o">.</span><span class="py">apply</span><span class="o">)(</span><span class="nv">ArticleForm</span><span class="o">.</span><span class="py">unapply</span><span class="o">))</span>
</code></pre></div></div>
<p>This mapping can also include <a href="https://www.playframework.com/documentation/2.8.x/ScalaForms#Defining-constraints-on-the-form">constraints</a> to validate the submitted values.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/ArticleForm.scala</span>
<span class="k">import</span> <span class="nn">play.api.data.Form</span>
<span class="k">import</span> <span class="nn">play.api.data.Forms.</span><span class="o">{</span><span class="n">mapping</span><span class="o">,</span> <span class="n">of</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">play.api.data.format.Formats._</span>
<span class="k">import</span> <span class="nn">play.api.data.validation.Constraints._</span>
<span class="c1">// In ArticleForm object</span>
<span class="k">val</span> <span class="nv">playForm</span><span class="k">:</span> <span class="kt">Form</span><span class="o">[</span><span class="kt">ArticleForm</span><span class="o">]</span> <span class="k">=</span> <span class="nc">Form</span><span class="o">(</span><span class="nf">mapping</span><span class="o">(</span>
<span class="s">"title"</span> <span class="o">-></span> <span class="n">of</span><span class="o">[</span><span class="kt">String</span><span class="o">].</span><span class="py">verifying</span><span class="o">(</span><span class="nf">minLength</span><span class="o">(</span><span class="mi">5</span><span class="o">)),</span>
<span class="s">"text"</span> <span class="o">-></span> <span class="n">of</span><span class="o">[</span><span class="kt">String</span><span class="o">],</span>
<span class="o">)(</span><span class="nv">ArticleForm</span><span class="o">.</span><span class="py">apply</span><span class="o">)(</span><span class="nv">ArticleForm</span><span class="o">.</span><span class="py">unapply</span><span class="o">))</span>
</code></pre></div></div>
<p>Missing fields and unfulfilled constraints generate error messages. Those are useful to inform users.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Action</span> <span class="o">{</span> <span class="k">implicit</span> <span class="n">request</span> <span class="k">=></span>
<span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">playForm</span>
<span class="o">.</span><span class="py">bindFromRequest</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span>
<span class="o">{</span> <span class="n">formWithErrors</span> <span class="k">=></span> <span class="nc">BadRequest</span><span class="o">(</span><span class="s">"Bad"</span><span class="o">)</span> <span class="o">},</span>
<span class="o">{</span> <span class="n">form</span> <span class="k">=></span> <span class="nc">Ok</span><span class="o">(</span><span class="s">"Good"</span><span class="o">)</span> <span class="o">}</span>
<span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>More on this bellow, but first we need to define how users send us information.</p>
<h2 id="endpoints">Endpoints</h2>
<p>ReST imposes strict conventions that are easy to follow.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
POST /articles controllers.ArticlesController.createArticle
GET /articles controllers.ArticlesController.listArticles
GET /articles/:id controllers.ArticlesController.showArticle(id: java.util.UUID)
PUT /articles/:id controllers.ArticlesController.updateArticle(id: java.util.UUID)
PATCH /articles/:id controllers.ArticlesController.updateArticle(id: java.util.UUID)
DELETE /articles/:id controllers.ArticlesController.deleteArticle(id: java.util.UUID)
</code></pre></div></div>
<p>Those endpoints are perfect for an API but limiting for a UI. Adding two more endpoints removes the need to shoehorn forms on the other views.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
GET /articles/new controllers.ArticlesController.newArticle
GET /articles/:id/edit controllers.ArticlesController.editArticle(id: java.util.UUID)
</code></pre></div></div>
<p>Beware, Play uses the first route that matches a URL. <code class="language-plaintext highlighter-rouge">/articles/new</code> must be above all <code class="language-plaintext highlighter-rouge">/articles/:id</code> routes to avoid unwanted surprises.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
GET /articles/new controllers.ArticlesController.newArticle
GET /articles/:id/edit controllers.ArticlesController.editArticle(id: java.util.UUID)
POST /articles controllers.ArticlesController.createArticle
GET /articles controllers.ArticlesController.listArticles
GET /articles/:id controllers.ArticlesController.showArticle(id: java.util.UUID)
PUT /articles/:id controllers.ArticlesController.updateArticle(id: java.util.UUID)
PATCH /articles/:id controllers.ArticlesController.updateArticle(id: java.util.UUID)
DELETE /articles/:id controllers.ArticlesController.deleteArticle(id: java.util.UUID)
</code></pre></div></div>
<p>Each of these methods needs to be defined.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/ArticlesController.scala</span>
<span class="k">package</span> <span class="nn">controllers</span>
<span class="k">import</span> <span class="nn">java.util.UUID</span>
<span class="k">import</span> <span class="nn">javax.inject._</span>
<span class="k">import</span> <span class="nn">play.api.mvc._</span>
<span class="nd">@Singleton</span>
<span class="k">class</span> <span class="nc">ArticlesController</span> <span class="nd">@Inject</span><span class="o">()(</span>
<span class="k">val</span> <span class="nv">controllerComponents</span><span class="k">:</span> <span class="kt">ControllerComponents</span>
<span class="o">)</span> <span class="k">extends</span> <span class="nc">BaseController</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">newArticle</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="s">"newArticle"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">editArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"editArticle - ${id}"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">createArticle</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="s">"createArticle"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">listArticles</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="s">"listArticles"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">showArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="s">"showArticle - ${id}"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">updateArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"updateArticle - ${id}"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">deleteArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"deleteArticle - ${id}"</span><span class="o">))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Half of those methods are very like those in the <a href="/blog/2020/04/01/playing-with-scala-rest-api.html">ReSTful API post</a>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/ArticlesController.scala</span>
<span class="c1">// In ArticlesController class</span>
<span class="k">val</span> <span class="nv">store</span> <span class="k">=</span> <span class="nv">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">UUID</span>, <span class="kt">models.Article</span><span class="o">]</span>
<span class="k">def</span> <span class="nf">listArticles</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="nc">Ok</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">listArticles</span><span class="o">(</span><span class="nv">store</span><span class="o">.</span><span class="py">values</span><span class="o">))</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">showArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span><span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">listArticles</span><span class="o">))</span> <span class="o">{</span> <span class="n">article</span> <span class="k">=></span>
<span class="nc">Ok</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">showArticle</span><span class="o">(</span><span class="n">article</span><span class="o">))</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">deleteArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="nv">store</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">listArticles</span><span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The first difference is the use of <code class="language-plaintext highlighter-rouge">Redirect</code> instead of other status codes. The second is the use of <code class="language-plaintext highlighter-rouge">views.html.articles...</code>. This will display views found in the <code class="language-plaintext highlighter-rouge">/app/views/articles</code> folder. Before diving into those views, we will implement the remaining methods.</p>
<p>The <code class="language-plaintext highlighter-rouge">newArticle</code> and <code class="language-plaintext highlighter-rouge">editArticle</code> display HTML forms. The first has empty placeholders while the second should contain pre-populated ones.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/ArticlesController.scala</span>
<span class="c1">// In ArticlesController class</span>
<span class="k">def</span> <span class="nf">newArticle</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="k">implicit</span> <span class="n">request</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">playform</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">playForm</span>
<span class="nc">Ok</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">newArticle</span><span class="o">(</span><span class="n">playform</span><span class="o">))</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">editArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="k">implicit</span> <span class="n">request</span> <span class="k">=></span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span><span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">listArticles</span><span class="o">))</span> <span class="o">{</span> <span class="n">article</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">form</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">fromModel</span><span class="o">(</span><span class="n">article</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">playform</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">playForm</span><span class="o">.</span><span class="py">fill</span><span class="o">(</span><span class="n">form</span><span class="o">)</span>
<span class="nc">Ok</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">editArticle</span><span class="o">(</span><span class="n">article</span><span class="o">,</span> <span class="n">playform</span><span class="o">))</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Where the previous two show a form, the next two processes it. Once again, using <code class="language-plaintext highlighter-rouge">Form</code> simplifies the task.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/ArticlesController.scala</span>
<span class="c1">// In ArticlesController class</span>
<span class="k">def</span> <span class="nf">createArticle</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="k">implicit</span> <span class="n">request</span> <span class="k">=></span>
<span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">playForm</span>
<span class="o">.</span><span class="py">bindFromRequest</span>
<span class="o">.</span><span class="py">fold</span><span class="o">({</span> <span class="n">formWithErrors</span> <span class="k">=></span>
<span class="nc">BadRequest</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">newArticle</span><span class="o">(</span><span class="n">formWithErrors</span><span class="o">))</span>
<span class="o">},</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">model</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">Article</span><span class="o">.</span><span class="py">fromForm</span><span class="o">(</span><span class="n">form</span><span class="o">)</span>
<span class="nv">store</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">model</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">model</span><span class="o">)</span>
<span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">showArticle</span><span class="o">(</span><span class="nv">model</span><span class="o">.</span><span class="py">id</span><span class="o">))</span>
<span class="o">})</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">updateArticle</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">UUID</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="k">implicit</span> <span class="n">request</span> <span class="k">=></span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span><span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">listArticles</span><span class="o">))</span> <span class="o">{</span> <span class="n">article</span> <span class="k">=></span>
<span class="nv">models</span><span class="o">.</span><span class="py">ArticleForm</span><span class="o">.</span><span class="py">playForm</span>
<span class="o">.</span><span class="py">bindFromRequest</span>
<span class="o">.</span><span class="py">fold</span><span class="o">({</span> <span class="n">formWithErrors</span> <span class="k">=></span>
<span class="nc">BadRequest</span><span class="o">(</span><span class="nv">views</span><span class="o">.</span><span class="py">html</span><span class="o">.</span><span class="py">articles</span><span class="o">.</span><span class="py">editArticle</span><span class="o">(</span><span class="n">article</span><span class="o">,</span> <span class="n">formWithErrors</span><span class="o">))</span>
<span class="o">},</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">model</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">Article</span><span class="o">.</span><span class="py">updated</span><span class="o">(</span><span class="n">article</span><span class="o">)(</span><span class="n">form</span><span class="o">)</span>
<span class="nv">store</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">model</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">model</span><span class="o">)</span>
<span class="nc">Redirect</span><span class="o">(</span><span class="nv">routes</span><span class="o">.</span><span class="py">ArticlesController</span><span class="o">.</span><span class="py">showArticle</span><span class="o">(</span><span class="nv">model</span><span class="o">.</span><span class="py">id</span><span class="o">))</span>
<span class="o">})</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This leaves the views.</p>
<h2 id="views">Views</h2>
<p>Twirl’s views start with their input.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@* In /app/views/articles/showArticle.scala.html *@
@(article: Article)
<span class="nt"><h1></span>Article<span class="nt"></h1></span>
<span class="nt"><p><strong></span>Id:<span class="nt"></strong></span> @article.id<span class="nt"></p></span>
<span class="nt"><p><strong></span>Title:<span class="nt"></strong></span> @article.title<span class="nt"></p></span>
<span class="nt"><p><strong></span>Text:<span class="nt"></strong></span> @article.text<span class="nt"></p></span>
<span class="nt"><p><strong></span>Created At:<span class="nt"></strong></span> @article.createdAt<span class="nt"></p></span>
<span class="nt"><p><strong></span>Updated At:<span class="nt"></strong></span> @article.updatedAt<span class="nt"></p></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.editArticle(article.id)"</span><span class="nt">></span>Edit<span class="nt"></a></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.listArticles"</span><span class="nt">></span>Back<span class="nt"></a></span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">@</code> is Twirl’s <a href="https://www.playframework.com/documentation/2.8.x/ScalaTemplates#Syntax:-the-magic-%E2%80%98@%E2%80%99-character">“magic character”</a>. It defines the input, accesses them, calls methods, loops through lists, and much more.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@* In /app/views/articles/listArticles.scala.html *@
@(articles: Iterable[Article])
<span class="nt"><h1></span>Articles<span class="nt"></h1></span>
<span class="nt"><table></span>
@for(article <span class="nt"><-</span> <span class="na">articles</span><span class="err">)</span> <span class="err">{</span>
<span class="err"><</span><span class="na">tr</span><span class="nt">></span>
<span class="nt"><td></span>@article.id<span class="nt"></td></span>
<span class="nt"><td></span>@article.title<span class="nt"></td></span>
<span class="nt"><td></span>@article.text<span class="nt"></td></span>
<span class="nt"><td></span>@article.createdAt<span class="nt"></td></span>
<span class="nt"><td></span>@article.updatedAt<span class="nt"></td></span>
<span class="nt"><td><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.showArticle(article.id)"</span><span class="nt">></span>Show<span class="nt"></a></td></span>
<span class="nt"><td><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.editArticle(article.id)"</span><span class="nt">></span>Edit<span class="nt"></a></td></span>
<span class="nt"></tr></span>
}
<span class="nt"></table></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.newArticle"</span><span class="nt">></span>New Article<span class="nt"></a></span>
</code></pre></div></div>
<p>Play has a few opinionated <a href="https://github.com/playframework/playframework/tree/master/core/play/src/main/scala/views/helper">Twirl helpers</a>. They make creating an HTML form with Play a breeze, but needs a bit of upfront work.</p>
<p>Most of the helpers display information to users like labels or errors. Play localizes those messages. This is a nice feature to have, but we can’t turn it off. This requires us to provide a <a href="https://www.playframework.com/documentation/2.8.x/api/scala/play/api/i18n/MessagesProvider.html"><code class="language-plaintext highlighter-rouge">MessagesProvider</code></a> to use those helpers. It is available in the <code class="language-plaintext highlighter-rouge">MessagesBaseController</code> instead of <code class="language-plaintext highlighter-rouge">BaseController</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/ArticlesController.scala</span>
<span class="nd">@Singleton</span>
<span class="k">class</span> <span class="nc">ArticlesController</span> <span class="nd">@Inject</span><span class="o">()(</span>
<span class="k">val</span> <span class="nv">controllerComponents</span><span class="k">:</span> <span class="kt">MessagesControllerComponents</span>
<span class="o">)</span> <span class="k">extends</span> <span class="nc">MessagesBaseController</span> <span class="o">{</span>
<span class="o">...</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Also, don’t forget to add the request as a view input.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@* in /app/views/articles/newArticle.scala.html *@
@(articleForm: Form[ArticleForm])(implicit request: MessagesRequestHeader)
<span class="nt"><h1></span>New Article<span class="nt"></h1></span>
@helper.form(routes.ArticlesController.createArticle) {
@helper.CSRF.formField
@helper.inputText(articleForm("title"))
@helper.textarea(articleForm("text"))
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"submit"</span> <span class="nt">/></span>
}
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.listArticles"</span><span class="nt">></span>Back<span class="nt"></a></span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">editArticle</code> view is the last to write. It should allow users to send <code class="language-plaintext highlighter-rouge">PUT</code>, <code class="language-plaintext highlighter-rouge">PATCH</code>, or <code class="language-plaintext highlighter-rouge">DELETE</code> requests. The only issue is that browsers can’t submit forms with these methods.</p>
<h2 id="browser-limitations">Browser Limitations</h2>
<blockquote>
<p><strong>method</strong></p>
<p>The HTTP method to submit the form with. Possible values:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">post</code>: The POST method; form data sent as the request body.</li>
<li><code class="language-plaintext highlighter-rouge">get</code>: The GET method; form data appended to the <code class="language-plaintext highlighter-rouge">action</code> URL with a <code class="language-plaintext highlighter-rouge">?</code> separator. Use this method when the form has no side-effects.</li>
<li><code class="language-plaintext highlighter-rouge">dialog</code>: When the form is inside a <code class="language-plaintext highlighter-rouge"><dialog></code>, closes the dialog on submission.</li>
</ul>
<p><a href="https://developer.mozilla.org/en-US/docs/Web/HTML/Element/form#attr-method">developer.mozilla.org</a></p>
</blockquote>
<p>Browsers can’t properly interact with ReSTful APIs. We could change the routes to <code class="language-plaintext highlighter-rouge">POST</code>s. We could send the request with JavaScript. But following what other frameworks do seems smarter.</p>
<p><a href="https://guides.rubyonrails.org/form_helpers.html#how-do-forms-with-patch-put-or-delete-methods-work-questionmark">Rails <code class="language-plaintext highlighter-rouge">POST</code> a form with a hidden <code class="language-plaintext highlighter-rouge">_method</code> input</a>. Many frameworks do the same. Unfortunately, Play doesn’t support this, but there is a library that does.</p>
<p><a href="https://github.com/plippe/play-form">play-form</a> is a small library. It contains two parts. A view that <code class="language-plaintext highlighter-rouge">POST</code>s a form with an extra query string argument and a <a href="https://www.playframework.com/documentation/2.8.x/ScalaHttpRequestHandlers"><code class="language-plaintext highlighter-rouge">RequestHandler</code></a> that extracts it. This allows browser requests to be ReSTful before they reach the router.</p>
<p>First, add the library as a dependency.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /build.sbt</span>
<span class="n">libraryDependencies</span> <span class="o">+=</span> <span class="s">"com.github.plippe"</span> <span class="o">%%</span> <span class="s">"play-form"</span> <span class="o">%</span> <span class="s">"2.8.1"</span>
</code></pre></div></div>
<p>Next, inform Play of the new request handler.</p>
<div class="language-hocon highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In /conf/application.conf</span><span class="w">
</span><span class="nl">play.http.requestHandler</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s2">"com.github.plippe.play.form.DefaultHttpRequestHandler"</span><span class="w">
</span></code></pre></div></div>
<p>Finally, use the included form instead of the helper one.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@* in /app/views/articles/editArticle.scala.html *@
@(article: Article, articleForm: Form[ArticleForm])(implicit request: MessagesRequestHeader)
@import com.github.plippe.play.form.form
<span class="nt"><h1></span>Edit Article<span class="nt"></h1></span>
@form(routes.ArticlesController.updateArticle(article.id)) {
@helper.CSRF.formField
@helper.inputText(articleForm(models.ArticleForm.TitleField))
@helper.textarea(articleForm(models.ArticleForm.TextField))
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"submit"</span> <span class="nt">/></span>
}
@form(routes.ArticlesController.deleteArticle(article.id)) {
@helper.CSRF.formField
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"submit"</span> <span class="na">value=</span><span class="s">"Delete"</span><span class="nt">/></span>
}
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.showArticle(article.id)"</span><span class="nt">></span>Show<span class="nt"></a></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"@routes.ArticlesController.listArticles"</span><span class="nt">></span>Back<span class="nt"></a></span>
</code></pre></div></div>
<p>And there we have it, a Twirl frontend that interacts with a ReSTful backend.</p>
<hr />
<p>There are a lot of pieces required to build applications users can interact with. Explaining each of them took a lot of words. Nothing complicated, but it was still long. Imagine how long it would have been if we had to cover a frontend framework too.</p>
<p>Hopefully, you can see the time saved by building a proof of concept using Twirl.</p>Philippe VinchonBuilding a single page application is a luxury not everyone can afford. If you are short on time, don’t learn Vue.js, React, or the other flavor of the month. Use Play Framework’s built-in templating engine, Twirl. It isn’t perfect but gets the job done.Playing With Scala - ReST API2020-04-01T00:00:00+00:002020-04-01T00:00:00+00:00https://plippe.github.io/blog/2020/04/01/playing-with-scala-rest-api<p>Play Framework is a simple way to get started with Scala. <a href="/blog/2020/03/01/playing-with-scala-hello-world.html">My first post</a> was a traditional “Hello, World!”. This one will show how to build a ReSTful API for a pet store. </p>
<p>As this is the second post in the series, don’t expect any advanced features. Shortcuts are taken to avoid <code class="language-plaintext highlighter-rouge">BodyParsers</code>, <code class="language-plaintext highlighter-rouge">Writeables</code>, and database interactions. Those will be covered in later posts.</p>
<p>The first step is to start a new project. The <a href="https://github.com/playframework/play-scala-seed.g8">official giter8 template</a> makes that step easy.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sbt new playframework/play-scala-seed.g8
</code></pre></div></div>
<p>The endpoints to read pets can be implemented next.</p>
<h2 id="reads">Reads</h2>
<p>The API’s consumers need a way to retrieve information. Two endpoints are required. The first list all pets and the second a single one. </p>
<p>The easiest way to start is by implementing the routes.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
GET /pets controllers.PetsController.get
GET /pets/:id controllers.PetsController.getById(id: String)
</code></pre></div></div>
<p>The first line is straight forward. It has a static URI. The second is a bit more complex. The placeholder <code class="language-plaintext highlighter-rouge">:id</code> will extract the segment and use it as an argument.</p>
<p>The compiler will now expect a <code class="language-plaintext highlighter-rouge">PetsController</code> class with the two methods.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="k">package</span> <span class="nn">controllers</span>
<span class="k">import</span> <span class="nn">javax.inject.Inject</span>
<span class="k">import</span> <span class="nn">play.api.mvc.</span><span class="o">{</span><span class="nc">BaseController</span><span class="o">,</span> <span class="nc">ControllerComponents</span><span class="o">}</span>
<span class="k">class</span> <span class="nc">PetsController</span> <span class="nd">@Inject</span><span class="o">()(</span>
<span class="k">val</span> <span class="nv">controllerComponents</span><span class="k">:</span> <span class="kt">ControllerComponents</span>
<span class="o">)</span> <span class="k">extends</span> <span class="nc">BaseController</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">get</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="s">"get"</span><span class="o">))</span>
<span class="k">def</span> <span class="nf">getById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span><span class="o">(</span><span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"getById(${id})"</span><span class="o">))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Extending Play’s <code class="language-plaintext highlighter-rouge">BaseController</code> helps keep the controllers more concise. The trait requires a <code class="language-plaintext highlighter-rouge">controllerComponents</code> defined but offers a lot in return. The simplest way to provide the variable is with dependency injection. By default, Play handles that with <a href="https://www.playframework.com/documentation/2.8.x/ScalaDependencyInjection">Guice</a>.</p>
<p>The methods currently return some text with a 200 status code, <code class="language-plaintext highlighter-rouge">Ok()</code>. To properly defined them, the pets need to be accessible from a data source. </p>
<p>To keep this post short, and leave room for other posts, a <code class="language-plaintext highlighter-rouge">Map</code> will be used instead of a proper database. Beware, compiling the application will remove all values from the <code class="language-plaintext highlighter-rouge">Map</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">val</span> <span class="nv">store</span> <span class="k">=</span> <span class="nv">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">String</span>, <span class="kt">models.Pet</span><span class="o">]</span>
</code></pre></div></div>
<p>This would be a good time to define the <code class="language-plaintext highlighter-rouge">Pet</code> class.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/Pet.scala</span>
<span class="k">package</span> <span class="nn">models</span>
<span class="k">case</span> <span class="k">class</span> <span class="nc">Pet</span><span class="o">(</span>
<span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">tag</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">String</span><span class="o">],</span>
<span class="o">)</span>
</code></pre></div></div>
<p>An implementation of <a href="https://www.playframework.com/documentation/2.8.x/ScalaJsonCombinators#Writes">Play JSON Writes</a> is required for the server to write the class in the JSON format.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/Pet.scala</span>
<span class="k">import</span> <span class="nn">play.api.libs.json.Json</span>
<span class="k">object</span> <span class="nc">Pet</span> <span class="o">{</span>
<span class="k">implicit</span> <span class="k">def</span> <span class="nf">petPlayJsonWrites</span> <span class="k">=</span> <span class="nv">Json</span><span class="o">.</span><span class="py">writes</span><span class="o">[</span><span class="kt">Pet</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>With everything defined, the methods can properly be implemented.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">def</span> <span class="nf">get</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">models</span> <span class="k">=</span> <span class="nv">store</span><span class="o">.</span><span class="py">values</span>
<span class="k">val</span> <span class="nv">json</span> <span class="k">=</span> <span class="nv">Json</span><span class="o">.</span><span class="py">toJson</span><span class="o">(</span><span class="n">models</span><span class="o">)</span>
<span class="nc">Ok</span><span class="o">(</span><span class="n">json</span><span class="o">)</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">getById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="nv">Json</span><span class="o">.</span><span class="py">toJson</span><span class="o">[</span><span class="kt">models.Pet</span><span class="o">])</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span><span class="nc">NotFound</span><span class="o">(</span><span class="n">s</span><span class="s">"Pet not found: ${id}"</span><span class="o">))(</span><span class="nc">Ok</span><span class="o">(</span><span class="k">_</span><span class="o">))</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The next step is to add the upsert operations.</p>
<h2 id="upserts">Upserts</h2>
<p>Two upsert methods exist. The first is to insert new pets and the second is to update an existing one. </p>
<p>Once again, the implementation process is easiest by starting with the routes.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
POST /pets controllers.PetsController.post
PUT /pets/:id controllers.PetsController.putById(id: String)
</code></pre></div></div>
<p>The payloads can’t be a <code class="language-plaintext highlighter-rouge">Pet</code>. This would require an identifier to create pets. Furthermore, updates would receive two, possibly different, values. A new type would solve those issues.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/PetForm.scala</span>
<span class="k">package</span> <span class="nn">models</span>
<span class="k">case</span> <span class="k">class</span> <span class="nc">PetForm</span><span class="o">(</span>
<span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span>
<span class="n">tag</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">String</span><span class="o">],</span>
<span class="o">)</span>
</code></pre></div></div>
<p>To mirror the JSON Writes for outputs, Play has <a href="https://www.playframework.com/documentation/2.8.x/ScalaJsonCombinators#Reads">JSON Reads</a> for inputs.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/models/PetForm.scala</span>
<span class="k">import</span> <span class="nn">play.api.libs.json.Json</span>
<span class="k">object</span> <span class="nc">PetForm</span> <span class="o">{</span>
<span class="k">implicit</span> <span class="k">def</span> <span class="nf">petFormPlayJsonReads</span> <span class="k">=</span> <span class="nv">Json</span><span class="o">.</span><span class="py">reads</span><span class="o">[</span><span class="kt">PetForm</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The bodies are accessible straight from the request. First as JSON and then converted to the <code class="language-plaintext highlighter-rouge">PetForm</code> type. <code class="language-plaintext highlighter-rouge">Either</code> is a simple way to handle the errors. Failures are kept on the left and successes on the right. Those need to be merged at the end.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">val</span> <span class="nv">missingContentType</span> <span class="k">=</span> <span class="nc">UnprocessableEntity</span><span class="o">(</span><span class="s">"Expected 'Content-Type' set to 'application/json'"</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">missingPetForm</span> <span class="k">=</span> <span class="nc">UnprocessableEntity</span><span class="o">(</span><span class="s">"Expected content to contain a pet form"</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">post</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="n">req</span> <span class="k">=></span>
<span class="nv">req</span><span class="o">.</span><span class="py">body</span><span class="o">.</span><span class="py">asJson</span>
<span class="o">.</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingContentType</span><span class="o">)</span>
<span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">asOpt</span><span class="o">[</span><span class="kt">models.PetForm</span><span class="o">].</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingPetForm</span><span class="o">))</span>
<span class="o">.</span><span class="py">map</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span> <span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"post - ${form}"</span><span class="o">)</span> <span class="o">}</span>
<span class="o">.</span><span class="py">merge</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">putById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="n">req</span> <span class="k">=></span>
<span class="nv">req</span><span class="o">.</span><span class="py">body</span><span class="o">.</span><span class="py">asJson</span>
<span class="o">.</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingContentType</span><span class="o">)</span>
<span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">asOpt</span><span class="o">[</span><span class="kt">models.PetForm</span><span class="o">].</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingPetForm</span><span class="o">))</span>
<span class="o">.</span><span class="py">flatMap</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span> <span class="nc">Ok</span><span class="o">(</span><span class="n">s</span><span class="s">"putById(${id}) - ${form}"</span><span class="o">)</span> <span class="o">}</span>
<span class="o">.</span><span class="py">merge</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The final hurdle is related to the <code class="language-plaintext highlighter-rouge">Map</code>. The current immutable value makes it impossible to insert or update pets. The store must be set to a mutable type.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">val</span> <span class="nv">store</span> <span class="k">=</span> <span class="nv">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">Map</span><span class="o">.</span><span class="py">empty</span><span class="o">[</span><span class="kt">String</span>, <span class="kt">models.Pet</span><span class="o">]</span>
</code></pre></div></div>
<p>This allows the methods to be properly defined.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">def</span> <span class="nf">post</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="n">req</span> <span class="k">=></span>
<span class="nv">req</span><span class="o">.</span><span class="py">body</span><span class="o">.</span><span class="py">asJson</span>
<span class="o">.</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingContentType</span><span class="o">)</span>
<span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">asOpt</span><span class="o">[</span><span class="kt">models.PetForm</span><span class="o">].</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingPetForm</span><span class="o">))</span>
<span class="o">.</span><span class="py">map</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">id</span> <span class="k">=</span> <span class="nv">UUID</span><span class="o">.</span><span class="py">randomUUID</span><span class="o">().</span><span class="py">toString</span>
<span class="k">val</span> <span class="nv">model</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">Pet</span><span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="nv">form</span><span class="o">.</span><span class="py">name</span><span class="o">,</span> <span class="nv">form</span><span class="o">.</span><span class="py">tag</span><span class="o">)</span>
<span class="nv">store</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="n">model</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">json</span> <span class="k">=</span> <span class="nv">Json</span><span class="o">.</span><span class="py">toJson</span><span class="o">(</span><span class="n">model</span><span class="o">)</span>
<span class="nc">Created</span><span class="o">(</span><span class="n">json</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">.</span><span class="py">merge</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">putById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span> <span class="n">req</span> <span class="k">=></span>
<span class="nv">req</span><span class="o">.</span><span class="py">body</span><span class="o">.</span><span class="py">asJson</span>
<span class="o">.</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingContentType</span><span class="o">)</span>
<span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">asOpt</span><span class="o">[</span><span class="kt">models.PetForm</span><span class="o">].</span><span class="py">toRight</span><span class="o">(</span><span class="n">missingPetForm</span><span class="o">))</span>
<span class="o">.</span><span class="py">flatMap</span> <span class="o">{</span> <span class="n">form</span> <span class="k">=></span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">toRight</span><span class="o">(</span><span class="nc">NotFound</span><span class="o">(</span><span class="n">s</span><span class="s">"Pet not found: ${id}"</span><span class="o">))</span>
<span class="o">.</span><span class="py">map</span><span class="o">((</span><span class="k">_</span><span class="o">,</span> <span class="n">form</span><span class="o">))</span>
<span class="o">}</span>
<span class="o">.</span><span class="py">map</span> <span class="o">{</span> <span class="nf">case</span> <span class="o">(</span><span class="n">found</span><span class="o">,</span> <span class="n">form</span><span class="o">)</span> <span class="k">=></span>
<span class="k">val</span> <span class="nv">model</span> <span class="k">=</span> <span class="nv">models</span><span class="o">.</span><span class="py">Pet</span><span class="o">(</span><span class="nv">found</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="nv">form</span><span class="o">.</span><span class="py">name</span><span class="o">,</span> <span class="nv">form</span><span class="o">.</span><span class="py">tag</span><span class="o">)</span>
<span class="nv">store</span><span class="o">.</span><span class="py">update</span><span class="o">(</span><span class="nv">found</span><span class="o">.</span><span class="py">id</span><span class="o">,</span> <span class="n">model</span><span class="o">)</span>
<span class="nc">NoContent</span>
<span class="o">}</span>
<span class="o">.</span><span class="py">merge</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The final step is to implement the delete operation.</p>
<h2 id="delete">Delete</h2>
<p>One last time, starting with the routes.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># In /conf/routes
DELETE /pets/:id controllers.PetsController.deleteById(id: String)
</code></pre></div></div>
<p>And finishing with the method.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// In /app/controllers/PetsController.scala</span>
<span class="c1">// In PetsController class</span>
<span class="k">def</span> <span class="nf">deleteById</span><span class="o">(</span><span class="n">id</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="k">=</span> <span class="nc">Action</span> <span class="o">{</span>
<span class="nv">store</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="o">.</span><span class="py">fold</span><span class="o">(</span><span class="nc">NotFound</span><span class="o">(</span><span class="n">s</span><span class="s">"Pet not found: ${id}"</span><span class="o">))</span> <span class="o">{</span> <span class="k">_</span> <span class="k">=></span>
<span class="nv">store</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="n">id</span><span class="o">)</span>
<span class="nc">NoContent</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>With all endpoints implemented, the API can be used to keep track of pets.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># sbt run to start the server</span>
<span class="c"># Add new pet</span>
curl localhost:9000/pets <span class="se">\</span>
<span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="se">\</span>
<span class="nt">-X</span> POST <span class="se">\</span>
<span class="nt">-d</span> <span class="s1">'{"name": "Snoopy"}'</span>
<span class="c"># Show all pets</span>
curl localhost:9000/pets
<span class="c"># Update existing pet, don't forget to change the ${ID}</span>
curl localhost:9000/pets/<span class="k">${</span><span class="nv">ID</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="se">\</span>
<span class="nt">-X</span> PUT <span class="se">\</span>
<span class="nt">-d</span> <span class="s1">'{"name": "Snoopy", "tag": "Peanuts"}'</span>
<span class="c"># Show pet</span>
curl localhost:9000/pets/<span class="k">${</span><span class="nv">ID</span><span class="k">}</span>
<span class="c"># Delete pet</span>
curl localhost:9000/pets/<span class="k">${</span><span class="nv">ID</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-X</span> DELETE
</code></pre></div></div>
<hr />
<p>The Play Framework made it simple to build a ReSTful API by handling the routing, the JSON conversion, and the HTTP statuses. The only thing missing is a proper UI.</p>
<p>If you want to have a look at the full application, the source code is available on <a href="https://github.com/plippe/playing-with-scala-rest-api">GitHub</a>.</p>Philippe VinchonPlay Framework is a simple way to get started with Scala. My first post was a traditional “Hello, World!”. This one will show how to build a ReSTful API for a pet store. Playing With Scala - Hello, World!2020-03-01T00:00:00+00:002020-03-01T00:00:00+00:00https://plippe.github.io/blog/2020/03/01/playing-with-scala-hello-world<p>Scala is a language stuck between two worlds. The FP world attracts many talents, but the OOP one is often where people start. <a href="https://plippe.github.io/blog/hashtags/cats.html">My “taming cats” posts</a> covered <a href="https://typelevel.org/cats/">cats</a> for the first group. Bellow is the start of a few posts on the <a href="http://playframework.com/">Play Framework</a> for the second.</p>
<h2 id="play-framework">Play Framework</h2>
<p>The Play Framework is one of the simplest ways to create web applications in Scala. The MVC framework contains <a href="https://www.playframework.com/documentation/2.8.x/ScalaRouting">routing</a>, <a href="https://www.playframework.com/documentation/2.8.x/ScalaJson">JSON formatting</a>, <a href="https://www.playframework.com/documentation/2.8.x/ScalaForms">forms and validation</a>, <a href="https://www.playframework.com/documentation/2.8.x/ScalaSessionFlash">sessions</a>, database interactions (<a href="https://www.playframework.com/documentation/2.8.x/Anorm">Anorm</a>, <a href="https://www.playframework.com/documentation/2.8.x/Evolutions">Evolutions</a>), <a href="https://www.playframework.com/documentation/2.8.x/ScalaTemplates">templating</a>, <a href="https://www.playframework.com/documentation/2.8.x/ScalaDependencyInjection">dependency injection</a> and much more.</p>
<p>Furthermore, the Scala knowledge to write controllers is low. At most, understanding <code class="language-plaintext highlighter-rouge">Future</code> is helpful, but not required.</p>
<p>Let’s prove that with a simple example.</p>
<h2 id="giter8">Giter8</h2>
<p>Play Framework has a <a href="http://www.foundweekends.org/giter8/">giter8</a> template to get started.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">></span> sbt new playframework/play-scala-seed.g8
<span class="o">[</span>info] Loading global plugins from <span class="o">[</span>HOME]/.sbt/1.0/plugins
<span class="o">[</span>info] Set current project to code <span class="o">(</span><span class="k">in </span>build file:[PWD]<span class="o">)</span>
<span class="o">[</span>info] Set current project to code <span class="o">(</span><span class="k">in </span>build file:[PWD]<span class="o">)</span>
This template generates a Play Scala project
name <span class="o">[</span>play-scala-seed]:
organization <span class="o">[</span>com.example]:
Template applied <span class="k">in</span> <span class="o">[</span>PWD]/./play-scala-seed
<span class="o">></span> <span class="nb">cd </span>play-scala-seed
</code></pre></div></div>
<p>The template contains a working application with a route, a controller, a view, and tests for all those.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sbt run
</code></pre></div></div>
<p>Once the server has started, <a href="http://localhost:9000/">localhost:9000</a> should display Play’s welcome message. From here, implementing “Hello, World!” is easy.</p>
<h2 id="hello-world">Hello, World!</h2>
<p>The first step is to add a new route in the <code class="language-plaintext highlighter-rouge">/conf/routes</code> file. The line starts with an HTTP verb, a URI, and the method to call. </p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>GET /hello-world controllers.HomeController.helloWorld
</code></pre></div></div>
<p>The new endpoint will throw an error. </p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>value helloWorld is not a member of controllers.HomeController
</code></pre></div></div>
<p>Implementing the missing method will fix the issue. The <code class="language-plaintext highlighter-rouge">HomeController</code> is in the <code class="language-plaintext highlighter-rouge">/app/controllers/HomeController.scala</code> file. The following <code class="language-plaintext highlighter-rouge">helloWorld</code> definition will do the trick.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def helloWorld = Action(Ok("Hello, World!"))
</code></pre></div></div>
<p>Opening <a href="http://localhost:9000/hello-world">localhost:9000/hello-world</a> should now display “Hello, World!”.</p>
<hr />
<p>Play Framework has a great template. It simplifies the getting started process like the “Hello, World!” endpoint above. It isn’t impressive but gives a stable foundation to build on. </p>
<p>With tradition out of the way, the next post will be on ReST.</p>Philippe VinchonScala is a language stuck between two worlds. The FP world attracts many talents, but the OOP one is often where people start. My “taming cats” posts covered cats for the first group. Bellow is the start of a few posts on the Play Framework for the second.Amazon S3 Versioning2020-01-01T00:00:00+00:002020-01-01T00:00:00+00:00https://plippe.github.io/blog/2020/01/01/amazon-s3-versioning<p>Amazon S3 is a great way to host files. It is similar to Google Drive, Apple iCloud, and Microsoft OneDrive, but for developers. Files are uploaded into Buckets under specific Keys. They can then be downloaded from around the world. There is a little bit more to it, but that is the main gist.</p>
<p>Each new upload brings a risk. A new file or a new version of an existing file could be incompatible with its consumers. Depending on the coupling, this could cause outages.</p>
<p>Versioning is a great way to mitigate this.</p>
<h2 id="amazon-s3-versioning">Amazon S3 versioning</h2>
<p>Amazon S3 has a built-in versioning solution. It can be enabled in the bucket’s properties tab.</p>
<p><img src="https://plippe.github.io/blog/assets/images/posts/amazon/amazon-s3-versioning.png" alt="Amazon S3 versioning" /></p>
<p>Once enabled, objects are never overwritten. Uploading multiple files to the same Bucket and Key will create new versions. Amazon S3 will return the latest one if none is explicitly requested.</p>
<p><img src="https://plippe.github.io/blog/assets/images/posts/amazon/amazon-s3-versions.png" alt="Amazon S3 versions" /></p>
<p>Furthermore, objects are never deleted. When an attempt is made, a new version is added to the Bucket and Key pair. That version is flagged to be unavailable.</p>
<p><img src="https://plippe.github.io/blog/assets/images/posts/amazon/amazon-s3-version-deleted.png" alt="amazon-s3-version-deleted" /></p>
<p>While objects can’t be deleted, versions can. This offers a revert mechanism.</p>
<h2 id="reverting-changes">Reverting changes</h2>
<p>If an outage is related to an Amazon S3 file it can be quicker to revert to a previous version instead of generating a new one.</p>
<p>Selecting the bad version and deleting it can be done in 5 clicks on the AWS Console.</p>
<p><img src="https://plippe.github.io/blog/assets/images/posts/amazon/amazon-s3-delete.png" alt="amazon-s3-delete" /></p>
<p>Once the bad version is removed, consumers should start retrieving the good one instead.</p>
<p>This useful functionality doesn’t come cheap.</p>
<h2 id="limiting-costs">Limiting costs</h2>
<p>Storing every single version can be expensive. AWS will charge you for every Gigabyte used. This includes objects that are flagged as deleted. To avoid the ever-growing bill, old versions should be deleted or moved to a service like <a href="https://aws.amazon.com/glacier/">Amazon S3 Glacier</a>.</p>
<p>This can be automated with <a href="https://aws.amazon.com/blogs/developer/amazon-s3-lifecycle-management/">Amazon S3 Lifecycle</a>. Objects older than a given set of days can be automatically handled, but if you wish to keep more than the latest version some work is required.</p>
<p>A simple solution is to trigger an AWS Lambda when a new version is added. The function would delete, if needs be, older versions.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">aws-sdk</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">s3</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">S3</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">AMOUNT_TO_KEEP</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="k">async</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">bucket</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">s3</span><span class="p">.</span><span class="nx">bucket</span><span class="p">.</span><span class="nx">name</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">key</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">s3</span><span class="p">.</span><span class="nx">object</span><span class="p">.</span><span class="nx">key</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">listParams</span> <span class="o">=</span> <span class="p">{</span> <span class="na">Bucket</span><span class="p">:</span> <span class="nx">bucket</span><span class="p">,</span> <span class="na">Prefix</span><span class="p">:</span> <span class="nx">key</span> <span class="p">};</span>
<span class="kd">const</span> <span class="nx">listResponse</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">listObjectVersions</span><span class="p">(</span><span class="nx">listParams</span><span class="p">)</span>
<span class="p">.</span><span class="nx">promise</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">deleteParams</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">Bucket</span><span class="p">:</span> <span class="nx">bucket</span><span class="p">,</span>
<span class="na">Delete</span><span class="p">:</span> <span class="p">{</span>
<span class="na">Objects</span><span class="p">:</span> <span class="nx">listResponse</span><span class="p">.</span><span class="nx">Versions</span>
<span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">AMOUNT_TO_KEEP</span><span class="p">)</span>
<span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">v</span> <span class="o">=></span> <span class="p">({</span> <span class="na">Key</span><span class="p">:</span> <span class="nx">key</span><span class="p">,</span> <span class="na">VersionId</span><span class="p">:</span> <span class="nx">v</span><span class="p">.</span><span class="nx">VersionId</span> <span class="p">}))</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">await</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">deleteObjects</span><span class="p">(</span><span class="nx">deleteParams</span><span class="p">).</span><span class="nx">promise</span><span class="p">();</span>
<span class="p">};</span>
</code></pre></div></div>
<p>While this is a simple solution, it would still be expensive. The AWS Lambda would execute a <code class="language-plaintext highlighter-rouge">LIST</code> and <code class="language-plaintext highlighter-rouge">DELETE</code> operation for every <code class="language-plaintext highlighter-rouge">PUT</code>. While deletes are free, listing the content of a bucket is one of the most expensive requests.</p>
<p>To limit how often the <code class="language-plaintext highlighter-rouge">LIST</code> command is executed, the AWS Lambda should be triggered periodically with an <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/WhatIsCloudWatchEvents.html">Amazon CloudWatch Event</a>. The frequency depends on how often files are pushed.</p>
<p>The AWS Lambda should retrieve all versions in as few calls as possible. Reducing the amount is important to keep the Amazon S3 bill low.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">listObjectVersions</span> <span class="o">=</span>
<span class="k">async</span><span class="p">(</span><span class="nx">s3</span><span class="p">,</span> <span class="nx">bucket</span><span class="p">,</span> <span class="nx">prefix</span><span class="p">,</span> <span class="nx">keyMarker</span><span class="p">,</span> <span class="nx">versionIdMarker</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">Bucket</span><span class="p">:</span> <span class="nx">bucket</span><span class="p">,</span>
<span class="na">Prefix</span><span class="p">:</span> <span class="nx">prefix</span><span class="p">,</span>
<span class="na">MaxKeys</span><span class="p">:</span> <span class="mi">1000</span><span class="p">,</span>
<span class="na">KeyMarker</span><span class="p">:</span> <span class="nx">keyMarker</span><span class="p">,</span>
<span class="na">VersionIdMarker</span><span class="p">:</span> <span class="nx">versionIdMarker</span>
<span class="p">};</span>
<span class="kd">const</span> <span class="nx">response</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">listObjectVersions</span><span class="p">(</span><span class="nx">params</span><span class="p">).</span><span class="nx">promise</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">more</span> <span class="o">=</span> <span class="p">(</span><span class="o">!</span><span class="nx">response</span><span class="p">.</span><span class="nx">IsTruncated</span><span class="p">)</span>
<span class="p">?</span> <span class="p">[]</span>
<span class="p">:</span> <span class="k">await</span> <span class="nx">listObjectVersions</span><span class="p">(</span>
<span class="nx">s3</span><span class="p">,</span>
<span class="nx">bucket</span><span class="p">,</span>
<span class="nx">prefix</span><span class="p">,</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">NextKeyMarker</span><span class="p">,</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">NextVersionIdMarker</span>
<span class="p">);</span>
<span class="k">return</span> <span class="nx">response</span><span class="p">.</span><span class="nx">Versions</span><span class="p">.</span><span class="nx">concat</span><span class="p">(</span><span class="nx">more</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div></div>
<p>With all versions available, the deprecated ones should be extracted. The example bellow keeps only the most recent ones, but more logic could be included.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">groupVersionsByKeys</span> <span class="o">=</span> <span class="p">(</span><span class="nx">versions</span><span class="p">)</span> <span class="o">=></span>
<span class="nx">versions</span><span class="p">.</span><span class="nx">reduce</span><span class="p">((</span><span class="nx">accumulator</span><span class="p">,</span> <span class="nx">v</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">accumulator</span><span class="p">[</span><span class="nx">v</span><span class="p">.</span><span class="nx">Key</span><span class="p">]</span> <span class="o">=</span> <span class="nx">accumulator</span><span class="p">[</span><span class="nx">v</span><span class="p">.</span><span class="nx">Key</span><span class="p">]</span> <span class="o">||</span> <span class="p">[];</span>
<span class="nx">accumulator</span><span class="p">[</span><span class="nx">v</span><span class="p">.</span><span class="nx">Key</span><span class="p">].</span><span class="nx">push</span><span class="p">(</span><span class="nx">v</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">accumulator</span><span class="p">;</span>
<span class="p">},</span> <span class="p">{});</span>
<span class="kd">const</span> <span class="nx">extractVersionsToDelete</span> <span class="o">=</span> <span class="p">(</span><span class="nx">versions</span><span class="p">,</span> <span class="nx">amountToKeep</span><span class="p">)</span> <span class="o">=></span>
<span class="nx">versions</span>
<span class="p">.</span><span class="nx">sort</span><span class="p">((</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="o">=></span> <span class="nx">b</span><span class="p">.</span><span class="nx">LastModified</span> <span class="o">-</span> <span class="nx">a</span><span class="p">.</span><span class="nx">LastModified</span><span class="p">)</span>
<span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">amountToKeep</span><span class="p">);</span>
</code></pre></div></div>
<p>The extracted versions can then be deleted.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">deleteObjectVersions</span> <span class="o">=</span> <span class="k">async</span><span class="p">(</span><span class="nx">s3</span><span class="p">,</span> <span class="nx">bucket</span><span class="p">,</span> <span class="nx">versions</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">versions</span><span class="p">.</span><span class="nx">length</span> <span class="o">===</span> <span class="mi">0</span><span class="p">)</span> <span class="k">return</span><span class="p">;</span>
<span class="k">else</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">first</span> <span class="o">=</span> <span class="nx">versions</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">AWS_S3_MAX_KEYS</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">more</span> <span class="o">=</span> <span class="nx">versions</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="nx">AWS_S3_MAX_KEYS</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">Bucket</span><span class="p">:</span> <span class="nx">bucket</span><span class="p">,</span>
<span class="na">Delete</span><span class="p">:</span> <span class="p">{</span>
<span class="na">Objects</span><span class="p">:</span> <span class="nx">first</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">v</span> <span class="o">=></span> <span class="p">({</span>
<span class="na">Key</span><span class="p">:</span> <span class="nx">v</span><span class="p">.</span><span class="nx">key</span><span class="p">,</span>
<span class="na">VersionId</span><span class="p">:</span> <span class="nx">v</span><span class="p">.</span><span class="nx">VersionId</span>
<span class="p">}))</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">await</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">deleteObjects</span><span class="p">(</span><span class="nx">params</span><span class="p">).</span><span class="nx">promise</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">deleteObjectVersions</span><span class="p">(</span><span class="nx">s3</span><span class="p">,</span> <span class="nx">bucket</span><span class="p">,</span> <span class="nx">more</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
</code></pre></div></div>
<p>The AWS Lambda’s handler combines the functions defined above.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">aws-sdk</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">s3</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">S3</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">BUCKET</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">example-us-east-1</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">AMOUNT_TO_KEEP</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="k">async</span><span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">versions</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">listObjectVersions</span><span class="p">(</span><span class="nx">s3</span><span class="p">,</span> <span class="nx">BUCKET</span><span class="p">,</span> <span class="dl">''</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">byKeys</span> <span class="o">=</span> <span class="nx">groupVersionsByKeys</span><span class="p">(</span><span class="nx">versions</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">toDelete</span> <span class="o">=</span> <span class="nb">Object</span><span class="p">.</span><span class="nx">keys</span><span class="p">(</span><span class="nx">versionsByKeys</span><span class="p">)</span>
<span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">k</span> <span class="o">=></span> <span class="nx">extractVersionsToDelete</span><span class="p">(</span><span class="nx">byKeys</span><span class="p">[</span><span class="nx">k</span><span class="p">],</span> <span class="nx">AMOUNT_TO_KEEP</span><span class="p">))</span>
<span class="p">.</span><span class="nx">flat</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">deleteObjectVersions</span><span class="p">(</span><span class="nx">s3</span><span class="p">,</span> <span class="nx">BUCKET</span><span class="p">,</span> <span class="nx">toDelete</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div></div>
<p>With everything in place, all but the last 3 uploaded versions of a Key will be deleted from the <code class="language-plaintext highlighter-rouge">example-us-east-1</code> Bucket.</p>
<hr />
<p>Amazon S3 is a great solution to many problems. The built-in versioning offers a simple revert mechanism, but at a price. If the built-in lifecycle management doesn’t work for you, the snippets above should help you keep the bill affordable.</p>Philippe VinchonAmazon S3 is a great way to host files. It is similar to Google Drive, Apple iCloud, and Microsoft OneDrive, but for developers. Files are uploaded into Buckets under specific Keys. They can then be downloaded from around the world. There is a little bit more to it, but that is the main gist.Tic Tac Toe With Rust2019-12-01T00:00:00+00:002019-12-01T00:00:00+00:00https://plippe.github.io/blog/2019/12/01/tic-tac-toe-with-rust<blockquote>
<p><strong>Disclaimer</strong>: The solution bellow works, but there are many cut corners. This is because I am learning Rust. Don’t hesitate to improve my solution in the comments, or on <a href="https://github.com/plippe/tic-tac-toe-rust">GitHub</a>.</p>
</blockquote>
<p>This isn’t <a href="/blog/2019/01/01/taming-cats-state.html">the first time</a> I talk about <a href="https://en.wikipedia.org/wiki/Tic-tac-toe">Tic Tac Toe</a>. The game is a great way to familiarize oneself with variables, tests, loops, and functions. With all those elements it isn’t an easy project, but it isn’t hard either.</p>
<p>The simplest way to code this game is with a state machine.</p>
<h2 id="state-machine">State Machine</h2>
<p>First, the game starts. This is the best time to select the player that will begin and create an empty board. This leads to the next state where players take turns selecting an available cell on the board. This repeats until either a player wins or no available cell remains. This marks the end of the game.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">enum</span> <span class="n">State</span> <span class="p">{</span>
<span class="n">StartGame</span><span class="p">,</span>
<span class="nf">NextTurn</span><span class="p">(</span><span class="n">Player</span><span class="p">,</span> <span class="n">Board</span><span class="p">),</span>
<span class="nf">Won</span><span class="p">(</span><span class="n">Player</span><span class="p">),</span>
<span class="n">Draw</span><span class="p">,</span>
<span class="n">EndGame</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">run</span><span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="o">&</span><span class="n">State</span><span class="p">)</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="k">match</span> <span class="n">state</span> <span class="p">{</span>
<span class="nn">State</span><span class="p">::</span><span class="n">StartGame</span> <span class="k">=></span> <span class="nd">unimplemented!</span><span class="p">(),</span>
<span class="nn">State</span><span class="p">::</span><span class="nf">NextTurn</span><span class="p">(</span><span class="n">player</span><span class="p">,</span> <span class="n">board</span><span class="p">)</span> <span class="k">=></span> <span class="nd">unimplemented!</span><span class="p">(),</span>
<span class="nn">State</span><span class="p">::</span><span class="nf">Won</span><span class="p">(</span><span class="n">player</span><span class="p">)</span> <span class="k">=></span> <span class="nd">unimplemented!</span><span class="p">(),</span>
<span class="nn">State</span><span class="p">::</span><span class="n">Draw</span> <span class="k">=></span> <span class="nd">unimplemented!</span><span class="p">(),</span>
<span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span> <span class="k">=></span> <span class="nd">unimplemented!</span><span class="p">(),</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">state</span> <span class="o">=</span> <span class="nn">State</span><span class="p">::</span><span class="n">StartGame</span><span class="p">;</span>
<span class="k">while</span> <span class="n">state</span> <span class="o">!=</span> <span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span> <span class="p">{</span>
<span class="n">state</span> <span class="o">=</span> <span class="nf">turn</span><span class="p">(</span><span class="o">&</span><span class="n">state</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>With the broad strokes explained above, most states are easy to define.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">start_game</span><span class="p">()</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Starting a new game"</span><span class="p">);</span>
<span class="nn">State</span><span class="p">::</span><span class="nf">NextTurn</span><span class="p">(</span><span class="nn">Player</span><span class="p">::</span><span class="nf">first</span><span class="p">(),</span> <span class="nn">Board</span><span class="p">::</span><span class="nf">new</span><span class="p">())</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">won</span><span class="p">(</span><span class="n">player</span><span class="p">:</span> <span class="o">&</span><span class="n">Player</span><span class="p">)</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Game finished and {:?} won"</span><span class="p">,</span> <span class="n">player</span><span class="p">);</span>
<span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">draw</span><span class="p">()</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Game finished with a draw"</span><span class="p">);</span>
<span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">end_game</span><span class="p">()</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Game finished"</span><span class="p">);</span>
<span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The only real complexity is around a player’s turn.</p>
<h2 id="nextturn">NextTurn</h2>
<p>A player’s turn starts by requesting and capturing their input.</p>
<p>For a game to be interactive players need to be able to submit information. In the case of Tic Tac Toe, players must select the cell they wish to mark. The game needs to guide them during that process like drawing the board.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">itertools</span><span class="p">::</span><span class="n">Itertools</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">collections</span><span class="p">::</span><span class="n">HashMap</span><span class="p">;</span>
<span class="k">struct</span> <span class="nf">Board</span><span class="p">(</span><span class="n">HashMap</span><span class="o"><</span><span class="n">Coordinates</span><span class="p">,</span> <span class="n">Player</span><span class="o">></span><span class="p">)</span>
<span class="k">impl</span> <span class="n">ToString</span> <span class="k">for</span> <span class="n">Board</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">to_string</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">String</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">cell_size</span> <span class="o">=</span> <span class="mi">5</span><span class="p">;</span>
<span class="k">let</span> <span class="n">line_split</span> <span class="o">=</span> <span class="nd">vec!</span><span class="p">[</span><span class="s">"-"</span><span class="nf">.repeat</span><span class="p">(</span><span class="n">cell_size</span><span class="p">);</span> <span class="mi">3</span><span class="p">];</span>
<span class="p">(</span><span class="mi">0</span><span class="o">..=</span><span class="mi">3</span><span class="p">)</span><span class="nf">.map</span><span class="p">(|</span><span class="n">y</span><span class="p">|</span> <span class="p">{</span>
<span class="p">(</span><span class="mi">0</span><span class="o">..=</span><span class="mi">3</span><span class="p">)</span><span class="nf">.map</span><span class="p">(</span><span class="k">move</span> <span class="p">|</span><span class="n">x</span><span class="p">|</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">cell_value</span> <span class="o">=</span> <span class="k">self</span><span class="na">.0</span>
<span class="nf">.get</span><span class="p">(</span><span class="o">&</span><span class="n">Coordinates</span> <span class="p">{</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">})</span>
<span class="nf">.map_or</span><span class="p">(</span>
<span class="nd">format!</span><span class="p">(</span><span class="s">"{},{}"</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span>
<span class="p">|</span><span class="n">player</span><span class="p">|</span> <span class="nd">format!</span><span class="p">(</span><span class="s">"{:?}"</span><span class="p">,</span> <span class="n">player</span><span class="p">)</span>
<span class="p">);</span>
<span class="nd">format!</span><span class="p">(</span><span class="s">"{: ^1$}"</span><span class="p">,</span> <span class="n">cell_value</span><span class="p">,</span> <span class="n">cell_size</span><span class="p">)</span>
<span class="p">})</span>
<span class="nf">.collect</span><span class="p">()</span>
<span class="p">})</span>
<span class="nf">.intersperse</span><span class="p">(</span><span class="n">line_split</span><span class="p">)</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">row</span><span class="p">|</span> <span class="n">row</span><span class="nf">.join</span><span class="p">(</span><span class="s">"|"</span><span class="p">))</span>
<span class="nf">.join</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>With the board visible, players are less likely to submit invalid inputs. Less likely means validation is still required.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">regex</span><span class="p">::</span><span class="n">Regex</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">str</span><span class="p">::</span><span class="n">FromStr</span><span class="p">;</span>
<span class="nd">#[derive(PartialEq,</span> <span class="nd">Eq,</span> <span class="nd">Clone,</span> <span class="nd">Hash)]</span>
<span class="k">pub</span> <span class="k">struct</span> <span class="n">Coordinates</span> <span class="p">{</span>
<span class="k">pub</span> <span class="n">x</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">y</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">FromStr</span> <span class="k">for</span> <span class="n">Coordinates</span> <span class="p">{</span>
<span class="k">type</span> <span class="nb">Err</span> <span class="o">=</span> <span class="nb">String</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">from_str</span><span class="p">(</span><span class="n">s</span><span class="p">:</span> <span class="o">&</span><span class="nb">str</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">Self</span><span class="p">,</span> <span class="nn">Self</span><span class="p">::</span><span class="nb">Err</span><span class="o">></span> <span class="p">{</span>
<span class="nn">Regex</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="s">r"^(-?[0-9]+),(-?[0-9]+)$"</span><span class="p">)</span>
<span class="nf">.unwrap</span><span class="p">()</span>
<span class="nf">.captures</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="nf">.and_then</span><span class="p">(|</span><span class="n">cap</span><span class="p">|</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">x</span> <span class="o">=</span> <span class="n">cap</span><span class="nf">.get</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="nf">.and_then</span><span class="p">(|</span><span class="n">m</span><span class="p">|</span> <span class="n">m</span><span class="nf">.as_str</span><span class="p">()</span><span class="nf">.parse</span><span class="p">()</span><span class="nf">.ok</span><span class="p">());</span>
<span class="k">let</span> <span class="n">y</span> <span class="o">=</span> <span class="n">cap</span><span class="nf">.get</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span><span class="nf">.and_then</span><span class="p">(|</span><span class="n">m</span><span class="p">|</span> <span class="n">m</span><span class="nf">.as_str</span><span class="p">()</span><span class="nf">.parse</span><span class="p">()</span><span class="nf">.ok</span><span class="p">());</span>
<span class="k">match</span> <span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span> <span class="p">{</span>
<span class="p">(</span><span class="nf">Some</span><span class="p">(</span><span class="n">x</span><span class="p">),</span> <span class="nf">Some</span><span class="p">(</span><span class="n">y</span><span class="p">))</span> <span class="k">=></span> <span class="nf">Some</span><span class="p">((</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)),</span>
<span class="mi">_</span> <span class="k">=></span> <span class="nb">None</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="nf">.map</span><span class="p">(|(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)|</span> <span class="n">Coordinates</span> <span class="p">{</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">})</span>
<span class="nf">.ok_or</span><span class="p">(</span><span class="s">"Coordinates can't be parsed"</span><span class="nf">.to_string</span><span class="p">())</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">Board</span> <span class="p">{</span>
<span class="k">pub</span> <span class="k">fn</span> <span class="nf">insert</span><span class="p">(</span>
<span class="o">&</span><span class="k">self</span><span class="p">,</span>
<span class="n">coordinates</span><span class="p">:</span> <span class="o">&</span><span class="n">Coordinates</span><span class="p">,</span>
<span class="n">player</span><span class="p">:</span> <span class="o">&</span><span class="n">Player</span><span class="p">,</span>
<span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">Board</span><span class="p">,</span> <span class="nb">String</span><span class="o">></span> <span class="p">{</span>
<span class="k">if</span> <span class="n">coordinates</span><span class="py">.x</span> <span class="o"><</span> <span class="mi">0</span> <span class="p">||</span> <span class="n">coordinates</span><span class="py">.x</span> <span class="o">></span> <span class="mi">3</span> <span class="p">||</span>
<span class="n">coordinates</span><span class="py">.y</span> <span class="o"><</span> <span class="mi">0</span> <span class="p">||</span> <span class="n">coordinates</span><span class="py">.y</span> <span class="o">></span> <span class="mi">3</span> <span class="p">{</span>
<span class="nf">Err</span><span class="p">(</span><span class="s">"Out of bounds"</span><span class="nf">.to_string</span><span class="p">())</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="k">self</span><span class="na">.0</span><span class="nf">.contains_key</span><span class="p">(</span><span class="n">coordinates</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">Err</span><span class="p">(</span><span class="s">"Already defined"</span><span class="nf">.to_string</span><span class="p">())</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">hash</span> <span class="o">=</span> <span class="k">self</span><span class="na">.0</span><span class="nf">.clone</span><span class="p">();</span>
<span class="n">hash</span><span class="nf">.insert</span><span class="p">(</span><span class="n">coordinates</span><span class="nf">.clone</span><span class="p">(),</span> <span class="n">player</span><span class="nf">.clone</span><span class="p">());</span>
<span class="nf">Ok</span><span class="p">(</span><span class="nf">Board</span><span class="p">(</span><span class="n">hash</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Before moving to the next player, the updated board needs to be checked for win conditions. On a small board, like Tic Tac Toe, hard coding the rows, columns, and diagonals to check is a quick and dirty solution.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">impl</span> <span class="n">Board</span> <span class="p">{</span>
<span class="k">pub</span> <span class="k">fn</span> <span class="nf">is_won</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">bool</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">get</span> <span class="o">=</span> <span class="p">|</span><span class="n">x</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="nb">i8</span><span class="p">|</span> <span class="k">self</span><span class="py">.hash</span><span class="nf">.get</span><span class="p">(</span><span class="o">&</span><span class="n">Coordinates</span><span class="p">{</span><span class="n">x</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span><span class="mi">0</span><span class="p">});</span>
<span class="nd">vec!</span><span class="p">[</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)),</span>
<span class="p">(</span><span class="nf">get</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="nf">get</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)),</span>
<span class="p">]</span>
<span class="nf">.iter</span><span class="p">()</span>
<span class="nf">.flat_map</span><span class="p">(|</span><span class="n">abc</span><span class="p">|</span> <span class="k">match</span> <span class="n">abc</span> <span class="p">{</span>
<span class="p">(</span><span class="nf">Some</span><span class="p">(</span><span class="n">a</span><span class="p">),</span> <span class="nf">Some</span><span class="p">(</span><span class="n">b</span><span class="p">),</span> <span class="nf">Some</span><span class="p">(</span><span class="n">c</span><span class="p">))</span> <span class="k">=></span> <span class="nf">Some</span><span class="p">((</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">)),</span>
<span class="mi">_</span> <span class="k">=></span> <span class="nb">None</span><span class="p">,</span>
<span class="p">})</span>
<span class="nf">.any</span><span class="p">(|(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">)|</span> <span class="n">a</span> <span class="o">==</span> <span class="n">b</span> <span class="o">&&</span> <span class="n">b</span> <span class="o">==</span> <span class="n">c</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">pub</span> <span class="k">fn</span> <span class="nf">is_draw</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">bool</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.hash</span><span class="nf">.len</span><span class="p">()</span> <span class="o">==</span> <span class="mi">9</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>By composing everything described above, we can write a function to handle <code class="language-plaintext highlighter-rouge">NextTurn</code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">next_turn</span><span class="p">(</span><span class="n">player</span><span class="p">:</span> <span class="o">&</span><span class="n">Player</span><span class="p">,</span> <span class="n">board</span><span class="p">:</span> <span class="o">&</span><span class="n">Board</span><span class="p">)</span> <span class="k">-></span> <span class="n">State</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Player {:?}'s turn"</span><span class="p">,</span> <span class="n">player</span><span class="p">);</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span> <span class="n">board</span><span class="nf">.to_string</span><span class="p">());</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">""</span><span class="p">);</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Where would you like to play?"</span><span class="p">);</span>
<span class="nn">read_input</span><span class="p">::</span><span class="o"><</span><span class="n">Coordinates</span><span class="o">></span><span class="p">()</span>
<span class="nf">.and_then</span><span class="p">(|</span><span class="n">coordinates</span><span class="p">|</span> <span class="n">board</span><span class="nf">.insert</span><span class="p">(</span><span class="o">&</span><span class="n">coordinates</span><span class="p">,</span> <span class="o">&</span><span class="n">player</span><span class="p">))</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">new_board</span><span class="p">|</span>
<span class="k">if</span> <span class="n">new_board</span><span class="nf">.is_won</span><span class="p">()</span> <span class="p">{</span> <span class="nn">State</span><span class="p">::</span><span class="nf">Won</span><span class="p">(</span><span class="n">player</span><span class="nf">.clone</span><span class="p">())</span> <span class="p">}</span>
<span class="k">else</span> <span class="k">if</span> <span class="n">new_board</span><span class="nf">.is_draw</span><span class="p">()</span> <span class="p">{</span> <span class="nn">State</span><span class="p">::</span><span class="n">Draw</span> <span class="p">}</span>
<span class="k">else</span> <span class="p">{</span> <span class="nn">State</span><span class="p">::</span><span class="nf">NextTurn</span><span class="p">(</span><span class="n">player</span><span class="nf">.next</span><span class="p">(),</span> <span class="n">new_board</span><span class="p">)</span> <span class="p">}</span>
<span class="p">)</span>
<span class="nf">.unwrap_or_else</span><span class="p">(|</span><span class="n">e</span><span class="p">|</span> <span class="p">{</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Error: {}"</span><span class="p">,</span> <span class="n">e</span><span class="p">);</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Try again?"</span><span class="p">);</span>
<span class="k">match</span> <span class="nn">read_input</span><span class="p">::</span><span class="o"><</span><span class="nb">bool</span><span class="o">></span><span class="p">()</span><span class="nf">.unwrap_or</span><span class="p">(</span><span class="k">false</span><span class="p">)</span> <span class="p">{</span>
<span class="k">true</span> <span class="k">=></span> <span class="nn">State</span><span class="p">::</span><span class="nf">NextTurn</span><span class="p">(</span><span class="n">player</span><span class="nf">.clone</span><span class="p">(),</span> <span class="n">board</span><span class="nf">.clone</span><span class="p">()),</span>
<span class="k">false</span> <span class="k">=></span> <span class="nn">State</span><span class="p">::</span><span class="n">EndGame</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This completes the Tic Tac Toe game, but could it be better?</p>
<h2 id="m-n-k-game">M, n, k-game</h2>
<p>Tic Tac Toe is an m, n, k-game. M refers to the number of rows, n the number of columns, and k the number of marks needed to win. Other m, n, k-game exist, like <a href="https://en.wikipedia.org/wiki/Gomoku">Gomoku</a>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">struct</span> <span class="n">Game</span> <span class="p">{</span>
<span class="k">pub</span> <span class="n">min_x</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">max_x</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">min_y</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">max_y</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">goal</span><span class="p">:</span> <span class="nb">i8</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">Game</span> <span class="p">{</span>
<span class="k">pub</span> <span class="k">const</span> <span class="n">TIC_TAC_TOE</span><span class="p">:</span> <span class="n">Game</span> <span class="o">=</span> <span class="n">Game</span> <span class="p">{</span>
<span class="n">min_x</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span>
<span class="n">max_x</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="n">min_y</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span>
<span class="n">max_y</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="n">goal</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span>
<span class="p">};</span>
<span class="k">pub</span> <span class="k">const</span> <span class="n">GOMOKU</span><span class="p">:</span> <span class="n">Game</span> <span class="o">=</span> <span class="n">Game</span> <span class="p">{</span>
<span class="n">min_x</span><span class="p">:</span> <span class="o">-</span><span class="mi">7</span><span class="p">,</span>
<span class="n">max_x</span><span class="p">:</span> <span class="mi">7</span><span class="p">,</span>
<span class="n">min_y</span><span class="p">:</span> <span class="o">-</span><span class="mi">7</span><span class="p">,</span>
<span class="n">max_y</span><span class="p">:</span> <span class="mi">7</span><span class="p">,</span>
<span class="n">goal</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Changing our game to handle any board size and winning constraint isn’t hard. Most of the functions above are already compatible. Those that need the most attention are in the <code class="language-plaintext highlighter-rouge">Board</code>’s implementation.</p>
<p>Taking the <code class="language-plaintext highlighter-rouge">ToString</code> implementation as an example. The hard coded <code class="language-plaintext highlighter-rouge">cell_size</code> needs to handle any x and y. Similarly, <code class="language-plaintext highlighter-rouge">line_split</code> must use the number of columns instead of 3. Lastly, the ranges are no longer between 0 and 3.</p>
<p>Instead of detailing <a href="https://github.com/plippe/tic-tac-toe-rust">my solution</a>, I will let you come up with yours. You shouldn’t have any issues writing your m, n, k-game. It can take a few hours, especially to identify if a player won, but you can do it.</p>
<hr />
<p>I greatly enjoyed building this little game. It was just the right size to stay enjoyable. I will definitely build more games in the future. Possibly Chess, Poker, or Yahtzee. Any preference?</p>Philippe VinchonDisclaimer: The solution bellow works, but there are many cut corners. This is because I am learning Rust. Don’t hesitate to improve my solution in the comments, or on GitHub.Swagger Pet Store With Rust2019-08-01T00:00:00+00:002019-08-01T00:00:00+00:00https://plippe.github.io/blog/2019/08/01/swagger-pet-store-with-rust<blockquote>
<p><strong>Disclaimer:</strong> The solution bellow works, but there are many cut corners. This is because I am learning Rust. Don’t hesitate to improve my solution in the comments, or <a href="https://github.com/plippe/swagger-pet-store-rust">on GitHub</a>.</p>
</blockquote>
<p>Rust has been on my radar for a year, or two. I even wrote about it three times. The first post was about <a href="/blog/2018/05/01/cross-compiling-rust-with-docker.html">cross compiling Rust code on docker</a>. The other two were about <a href="/blog/2018/06/01/exercism-driven-learning.html">learning Rust with Exercism.io</a>.</p>
<p>Since then, I found many excuses that kept me from progressing, but this ends now. Say hello to my fourth Rust post.</p>
<p>To start, I will introduce Swagger, and its Pet Store example. Next, I will have a quick look at Rust’s web frameworks. Finally, I will create a web service that fulfills the Pet Store specification.</p>
<h2 id="pet-store">Pet Store</h2>
<p><a href="https://swagger.io/">Swagger</a> aims to reduce friction in building, and consuming APIs. Given an <a href="https://swagger.io/specification/">OpenAPI specification</a>, their tools can <a href="https://swagger.io/tools/swagger-codegen/">generate code</a>, <a href="https://swagger.io/tools/swagger-ui/">display documentation</a>, and <a href="https://swagger.io/tools/">more</a>.</p>
<p>Swagger offers a few examples to get started. Their most popular one is the <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/examples/v3.0/petstore.yaml">pet store</a>. This version describes a way to create new pets, list them, or show a single one.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl .../pets <span class="nt">-d</span> <span class="s1">'{"id": 1, "name": "foo", "tag": "bar"}'</span>
curl .../pets?limit<span class="o">=</span>25
curl .../pets/1
</code></pre></div></div>
<p>My first Rust project could be a hello world, a todo application, or an original idea. Instead, I will jump in the deep end, and build this pet store.</p>
<h2 id="web-servers">Web servers</h2>
<p><a href="https://www.arewewebyet.org/">Are we web yet?</a> lists two groups of Rust web frameworks.</p>
<p>The first contains <a href="https://hyper.rs/">Hyper</a>, and <a href="https://crates.io/crates/tiny_http">tiny_http</a>. They are low-level frameworks that handle connections, requests, and responses. All other functionalities are missing.</p>
<p>The second has higher-level frameworks: <a href="https://actix.rs/">Actix-web</a>, <a href="https://rocket.rs/">Rocket</a>, and <a href="https://www.arewewebyet.org/topics/frameworks/">many more</a>. Often built on top of the previous group, they centralize common features in a need package.</p>
<p>The pet store is quite a simple project. The biggest hurdles are the routing, and JSON handling. This makes neither group attractive. The first would need extra work, but the second would make it too easy.</p>
<p>As I am here to learn, and I like the name, I chose Hyper.</p>
<h2 id="hello-world">Hello world</h2>
<p>The first step is to create a new Rust application. The simplest way to do so is with <a href="https://doc.rust-lang.org/stable/cargo/">Cargo</a>. The <code class="language-plaintext highlighter-rouge">init</code> command creates a new Cargo package in an existing directory, while <code class="language-plaintext highlighter-rouge">new</code> creates the directory too.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Uncomment one of the following</span>
<span class="c"># cargo new <path></span>
<span class="c"># cargo init</span>
</code></pre></div></div>
<p>The command should create at least two new files: <code class="language-plaintext highlighter-rouge">Cargo.toml</code>, and <code class="language-plaintext highlighter-rouge">src/main.rs</code>. The first is the manifest, and the second is a simple hello world. <code class="language-plaintext highlighter-rouge">cargo run</code> will execute it.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cargo run
Compiling <name> v0.1.0 <span class="o">(</span><path><span class="o">)</span>
Finished dev <span class="o">[</span>unoptimized + debuginfo] target<span class="o">(</span>s<span class="o">)</span> <span class="k">in </span>0.33s
Running <span class="sb">`</span>target/debug/<name><span class="sb">`</span>
Hello, world!
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Hyper</code> has a good <a href="https://hyper.rs/guides/server/hello-world/">getting started guide</a>. In short, there is a new dependency in <code class="language-plaintext highlighter-rouge">Cargo.toml</code>.</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># In Cargo.toml</span>
<span class="nn">[dependencies]</span>
<span class="py">hyper</span> <span class="p">=</span> <span class="s">"0.12.33"</span>
</code></pre></div></div>
<p>And, <code class="language-plaintext highlighter-rouge">src/mains.rs</code> has more content.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cs"># In src/mains.rs</span>
<span class="k">extern</span> <span class="n">crate</span> <span class="n">hyper</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">hyper</span><span class="p">::</span><span class="nn">rt</span><span class="p">::</span><span class="n">Future</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">hyper</span><span class="p">::</span><span class="nn">service</span><span class="p">::</span><span class="n">service_fn_ok</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">hyper</span><span class="p">::{</span><span class="n">Body</span><span class="p">,</span> <span class="n">Request</span><span class="p">,</span> <span class="n">Response</span><span class="p">,</span> <span class="n">Server</span><span class="p">};</span>
<span class="k">fn</span> <span class="nf">hello_world</span><span class="p">(</span><span class="mi">_</span><span class="n">req</span><span class="p">:</span> <span class="n">Request</span><span class="o"><</span><span class="n">Body</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="n">Response</span><span class="o"><</span><span class="n">Body</span><span class="o">></span> <span class="p">{</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="s">"Hello, world!"</span><span class="p">))</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">addr</span> <span class="o">=</span> <span class="p">([</span><span class="mi">127</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="mi">3000</span><span class="p">)</span><span class="nf">.into</span><span class="p">();</span>
<span class="k">let</span> <span class="n">new_svc</span> <span class="o">=</span> <span class="p">||</span> <span class="nf">service_fn_ok</span><span class="p">(</span><span class="n">hello_world</span><span class="p">);</span>
<span class="k">let</span> <span class="n">server</span> <span class="o">=</span> <span class="nn">Server</span><span class="p">::</span><span class="nf">bind</span><span class="p">(</span><span class="o">&</span><span class="n">addr</span><span class="p">)</span>
<span class="nf">.serve</span><span class="p">(</span><span class="n">new_svc</span><span class="p">)</span>
<span class="nf">.map_err</span><span class="p">(|</span><span class="n">e</span><span class="p">|</span> <span class="nd">eprintln!</span><span class="p">(</span><span class="s">"server error: {}"</span><span class="p">,</span> <span class="n">e</span><span class="p">));</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"Listening on http://{}"</span><span class="p">,</span> <span class="n">addr</span><span class="p">);</span>
<span class="nn">hyper</span><span class="p">::</span><span class="nn">rt</span><span class="p">::</span><span class="nf">run</span><span class="p">(</span><span class="n">server</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">cargo run</code> will start the server on the port 3000.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl localhost:3000
Hello, world!
</code></pre></div></div>
<h2 id="router">Router</h2>
<p>Hyper doesn’t come with a built-in routing logic. <a href="https://hyper.rs/guides/server/echo/">The echo example</a> suggests to pattern match the request’s method and path.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">match</span> <span class="p">(</span><span class="n">req</span><span class="nf">.method</span><span class="p">(),</span> <span class="n">req</span><span class="nf">.uri</span><span class="p">()</span><span class="nf">.path</span><span class="p">())</span> <span class="p">{</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">GET</span><span class="p">,</span> <span class="s">"/"</span><span class="p">)</span> <span class="k">=></span> <span class="p">{</span>
<span class="o">*</span><span class="n">response</span><span class="nf">.body_mut</span><span class="p">()</span> <span class="o">=</span> <span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="s">"Try POSTing data to /echo"</span><span class="p">);</span>
<span class="p">},</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">POST</span><span class="p">,</span> <span class="s">"/echo"</span><span class="p">)</span> <span class="k">=></span> <span class="p">{</span>
<span class="c">// we'll be back</span>
<span class="p">},</span>
<span class="mi">_</span> <span class="k">=></span> <span class="p">{</span>
<span class="o">*</span><span class="n">response</span><span class="nf">.status_mut</span><span class="p">()</span> <span class="o">=</span> <span class="nn">StatusCode</span><span class="p">::</span><span class="n">NOT_FOUND</span><span class="p">;</span>
<span class="p">},</span>
<span class="p">};</span>
</code></pre></div></div>
<p>Pattern matching strings only work for static URLs. Those containing variables, like <code class="language-plaintext highlighter-rouge">/pets/{petId}</code>, need a bit more logic. Splitting the string on the slash character, <code class="language-plaintext highlighter-rouge">/</code>, returns matchable segments, and extractable variables.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="n">get_method</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">(</span><span class="n">req</span><span class="p">:</span> <span class="o">&</span><span class="n">Request</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="n">Method</span> <span class="p">{</span>
<span class="n">req</span><span class="nf">.method</span><span class="p">()</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="n">get_path_segments</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">(</span><span class="n">req</span><span class="p">:</span> <span class="o">&</span><span class="n">Request</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="nb">Vec</span><span class="o"><&</span><span class="nb">str</span><span class="o">></span> <span class="p">{</span>
<span class="n">req</span><span class="nf">.uri</span><span class="p">()</span><span class="nf">.path</span><span class="p">()</span><span class="nf">.trim_matches</span><span class="p">(</span><span class="sc">'/'</span><span class="p">)</span><span class="nf">.split</span><span class="p">(</span><span class="sc">'/'</span><span class="p">)</span><span class="nf">.collect</span><span class="p">()</span>
<span class="p">}</span>
<span class="k">match</span> <span class="p">(</span><span class="nf">get_method</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">),</span> <span class="nf">get_path_segments</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">)</span><span class="nf">.as_slice</span><span class="p">())</span> <span class="p">{</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">GET</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="s">"GET /pets"</span><span class="p">;</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span><span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">POST</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="s">"POST /pets"</span><span class="p">;</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span><span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">GET</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">,</span> <span class="n">pet_id</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="nd">format!</span><span class="p">(</span><span class="s">"GET /pets/{}"</span><span class="p">,</span> <span class="n">pet_id</span><span class="p">);</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span><span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="mi">_</span> <span class="k">=></span> <span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.status</span><span class="p">(</span><span class="nn">StatusCode</span><span class="p">::</span><span class="n">NOT_FOUND</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">empty</span><span class="p">()),</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The specification requires <code class="language-plaintext highlighter-rouge">pet_id</code> to be a <code class="language-plaintext highlighter-rouge">String</code>. Other types would need more code, but not in this example.</p>
<p>The last missing router pieces are the query arguments. <code class="language-plaintext highlighter-rouge">listPets</code> has a <code class="language-plaintext highlighter-rouge">limit</code> parameter, and another to paginate results. The latter isn’t in the specification but will be available in the <code class="language-plaintext highlighter-rouge">x-next</code> header.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">str</span><span class="p">::</span><span class="n">FromStr</span><span class="p">;</span>
<span class="k">fn</span> <span class="n">get_query_parameter</span><span class="o"><</span><span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">:</span> <span class="n">FromStr</span><span class="o">></span><span class="p">(</span>
<span class="n">req</span><span class="p">:</span> <span class="o">&</span><span class="n">Request</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">,</span>
<span class="n">parameter_name</span><span class="p">:</span> <span class="o">&</span><span class="nb">str</span><span class="p">,</span>
<span class="p">)</span> <span class="k">-></span> <span class="nb">Option</span><span class="o"><</span><span class="n">B</span><span class="o">></span> <span class="p">{</span>
<span class="n">req</span><span class="nf">.uri</span><span class="p">()</span>
<span class="nf">.query</span><span class="p">()</span>
<span class="nf">.unwrap_or</span><span class="p">(</span><span class="s">""</span><span class="p">)</span>
<span class="nf">.split</span><span class="p">(</span><span class="sc">'&'</span><span class="p">)</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">name_value</span><span class="p">|</span> <span class="n">name_value</span><span class="nf">.split</span><span class="p">(</span><span class="sc">'='</span><span class="p">)</span><span class="nf">.collect</span><span class="p">())</span>
<span class="nf">.flat_map</span><span class="p">(|</span><span class="n">name_value</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><&</span><span class="nb">str</span><span class="o">></span><span class="p">|</span> <span class="k">match</span> <span class="n">name_value</span><span class="nf">.as_slice</span><span class="p">()</span> <span class="p">{</span>
<span class="p">[</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">]</span> <span class="k">=></span> <span class="nd">vec!</span><span class="p">[(</span><span class="n">name</span><span class="nf">.to_string</span><span class="p">(),</span> <span class="n">value</span><span class="nf">.to_string</span><span class="p">())],</span>
<span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">=></span> <span class="nd">vec!</span><span class="p">[(</span><span class="n">name</span><span class="nf">.to_string</span><span class="p">(),</span> <span class="s">"true"</span><span class="nf">.to_string</span><span class="p">())],</span>
<span class="mi">_</span> <span class="k">=></span> <span class="nd">vec!</span><span class="p">[],</span>
<span class="p">})</span>
<span class="nf">.find</span><span class="p">(|(</span><span class="n">name</span><span class="p">,</span> <span class="mi">_</span><span class="p">)|</span> <span class="n">name</span> <span class="o">==</span> <span class="n">parameter_name</span><span class="p">)</span>
<span class="nf">.and_then</span><span class="p">(|(</span><span class="mi">_</span><span class="p">,</span> <span class="n">value</span><span class="p">)|</span> <span class="n">value</span><span class="py">.parse</span><span class="p">::</span><span class="o"><</span><span class="n">B</span><span class="o">></span><span class="p">()</span><span class="nf">.ok</span><span class="p">())</span>
<span class="p">}</span>
<span class="k">match</span> <span class="p">(</span><span class="nf">get_method</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">),</span> <span class="nf">get_path_segments</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">)</span><span class="nf">.as_slice</span><span class="p">())</span> <span class="p">{</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">GET</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">limit</span> <span class="o">=</span> <span class="nf">get_query_parameter</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">,</span> <span class="s">"limit"</span><span class="p">)</span>
<span class="nf">.unwrap_or</span><span class="p">(</span><span class="mi">25</span><span class="p">);</span>
<span class="k">let</span> <span class="n">offset</span> <span class="o">=</span> <span class="nf">get_query_parameter</span><span class="p">(</span><span class="o">&</span><span class="n">req</span><span class="p">,</span> <span class="s">"offset"</span><span class="p">)</span>
<span class="nf">.unwrap_or</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="k">let</span> <span class="n">x_next</span> <span class="o">=</span> <span class="nd">format!</span><span class="p">(</span>
<span class="s">"/pets?limit={}&offset={}"</span><span class="p">,</span>
<span class="n">limit</span><span class="p">,</span>
<span class="n">limit</span> <span class="o">+</span> <span class="n">offset</span><span class="p">,</span>
<span class="p">);</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="s">"GET /pets"</span><span class="p">;</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.header</span><span class="p">(</span><span class="s">"x-next"</span><span class="p">,</span> <span class="n">x_next</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">POST</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="s">"POST /pets"</span><span class="p">;</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span><span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">(</span><span class="o">&</span><span class="nn">Method</span><span class="p">::</span><span class="n">GET</span><span class="p">,</span> <span class="p">[</span><span class="s">"pets"</span><span class="p">,</span> <span class="n">pet_id</span><span class="p">])</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">body</span> <span class="o">=</span> <span class="nd">format!</span><span class="p">(</span><span class="s">"GET /pets/{}"</span><span class="p">,</span> <span class="n">pet_id</span><span class="p">);</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span><span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="p">}</span>
<span class="mi">_</span> <span class="k">=></span> <span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.status</span><span class="p">(</span><span class="nn">StatusCode</span><span class="p">::</span><span class="n">NOT_FOUND</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">empty</span><span class="p">()),</span>
<span class="p">}</span>
</code></pre></div></div>
<p>With the routing in place, I can look at returning proper responses, and reading incoming ones.</p>
<h2 id="json">JSON</h2>
<p>Before even looking at JSON, I can create structures to represent the models.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">Pet</span> <span class="p">{</span>
<span class="n">id</span><span class="p">:</span> <span class="nb">u64</span><span class="p">,</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">String</span><span class="p">,</span>
<span class="n">tag</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">String</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">Pets</span> <span class="p">{</span>
<span class="n">items</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">Pet</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">Error</span> <span class="p">{</span>
<span class="n">code</span><span class="p">:</span> <span class="nb">u32</span><span class="p">,</span>
<span class="n">message</span><span class="p">:</span> <span class="nb">String</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Writing my serializer is a bad move. I shouldn’t reinvent the wheel. Hyper might not ship with JSON support, but that isn’t an issue with <a href="https://serde.rs/">Serde</a> around.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">hyper</span><span class="p">::</span><span class="nn">http</span><span class="p">::</span><span class="nn">header</span><span class="p">::</span><span class="n">CONTENT_TYPE</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">serde</span><span class="p">::</span><span class="n">Serialize</span><span class="p">;</span>
<span class="nd">#[derive(Serialize)]</span>
<span class="k">struct</span> <span class="n">Pet</span> <span class="p">{</span>
<span class="n">id</span><span class="p">:</span> <span class="nb">u64</span><span class="p">,</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">String</span><span class="p">,</span>
<span class="n">tag</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">String</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">let</span> <span class="n">json</span> <span class="o">=</span> <span class="nn">serde_json</span><span class="p">::</span><span class="nf">to_string</span><span class="p">(</span><span class="o">&</span><span class="n">pet</span><span class="p">)</span><span class="nf">.unwrap</span><span class="p">();</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.header</span><span class="p">(</span><span class="n">CONTENT_TYPE</span><span class="p">,</span> <span class="s">"application/json"</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">json</span><span class="p">))</span>
</code></pre></div></div>
<p>With the output returning valid JSON, the next part is about the input.</p>
<p>Hyper represents payloads as a stream of bytes. It reads them asynchronously making their content available in <code class="language-plaintext highlighter-rouge">Future</code>s. This affects return types. It forces <code class="language-plaintext highlighter-rouge">Box<Future<... Response<A> ...></code> instead of <code class="language-plaintext highlighter-rouge">Result<Response<A>></code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">futures</span><span class="p">::{</span><span class="n">Future</span><span class="p">,</span> <span class="n">Stream</span><span class="p">};</span>
<span class="k">type</span> <span class="n">FutureResponse</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="o">=</span>
<span class="nb">Box</span><span class="o"><</span><span class="n">dyn</span> <span class="n">Future</span><span class="o"><</span><span class="n">Item</span> <span class="o">=</span> <span class="n">Response</span><span class="o"><</span><span class="n">A</span><span class="o">></span><span class="p">,</span> <span class="n">Error</span> <span class="o">=</span> <span class="nn">hyper</span><span class="p">::</span><span class="n">Error</span><span class="o">></span> <span class="o">+</span> <span class="nb">Send</span><span class="o">></span><span class="p">;</span>
<span class="k">let</span> <span class="n">res</span><span class="p">:</span> <span class="n">FutureResponse</span> <span class="o">=</span> <span class="nn">Box</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">req</span><span class="nf">.into_body</span><span class="p">()</span>
<span class="nf">.concat2</span><span class="p">()</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">body</span><span class="p">|</span> <span class="p">{</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.status</span><span class="p">(</span><span class="nn">StatusCode</span><span class="p">::</span><span class="n">OK</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">body</span><span class="p">))</span>
<span class="nf">.unwrap</span><span class="p">()</span>
<span class="p">}));</span>
</code></pre></div></div>
<p>With the <code class="language-plaintext highlighter-rouge">body</code> available, Serde can deserialize it.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">serde</span><span class="p">::{</span><span class="n">Deserialize</span><span class="p">,</span> <span class="n">Serialize</span><span class="p">};</span>
<span class="nd">#[derive(Deserialize,</span> <span class="nd">Serialize)]</span>
<span class="k">struct</span> <span class="n">Pet</span> <span class="p">{</span>
<span class="n">id</span><span class="p">:</span> <span class="nb">u64</span><span class="p">,</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">String</span><span class="p">,</span>
<span class="n">tag</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">String</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">let</span> <span class="n">res</span> <span class="o">=</span> <span class="n">req</span>
<span class="nf">.into_body</span><span class="p">()</span>
<span class="nf">.concat2</span><span class="p">()</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">body</span><span class="p">|</span> <span class="nn">serde_json</span><span class="p">::</span><span class="nn">from_slice</span><span class="p">::</span><span class="o"><</span><span class="n">Pet</span><span class="o">></span><span class="p">(</span><span class="o">&</span><span class="n">body</span><span class="p">)</span><span class="nf">.unwrap</span><span class="p">())</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">pet</span><span class="p">|</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">json</span> <span class="o">=</span> <span class="nn">serde_json</span><span class="p">::</span><span class="nf">to_string</span><span class="p">(</span><span class="o">&</span><span class="n">pet</span><span class="p">)</span><span class="nf">.unwrap</span><span class="p">();</span>
<span class="nn">Response</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.header</span><span class="p">(</span><span class="n">CONTENT_TYPE</span><span class="p">,</span> <span class="s">"application/json"</span><span class="p">)</span>
<span class="nf">.body</span><span class="p">(</span><span class="nn">Body</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">json</span><span class="p">))</span>
<span class="nf">.unwrap</span><span class="p">()</span>
<span class="p">});</span>
<span class="nn">Box</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">res</span><span class="p">)</span>
</code></pre></div></div>
<p>With all the bricks defined, the Swagger specification is no longer hard to implement.</p>
<hr />
<p>And this concludes my first real Rust project. The full code is available on <a href="https://github.com/plippe/swagger-pet-store-rust">GitHub</a>, but I wouldn’t recommend it. It is quite unsafe, with <code class="language-plaintext highlighter-rouge">unwrap</code> in many places. The variable ownership and references are probably wrong. It also lacks basic functionalities, like database interactions.</p>
<p>I seem to have plenty more to learn, and that is exciting. I look forward to improving this project as I get better.</p>Philippe VinchonDisclaimer: The solution bellow works, but there are many cut corners. This is because I am learning Rust. Don’t hesitate to improve my solution in the comments, or on GitHub.Monitoring AWS IP Availability2019-07-01T00:00:00+00:002019-07-01T00:00:00+00:00https://plippe.github.io/blog/2019/07/01/monitoring-aws-ip-availability<p>Running virtual private cloud, or VPC, on AWS is common, and even required in some cases.</p>
<p>VPCs are virtual networks. They logically group AWS resources. Those instances are either connected to the internet or not thanks to subnets. All the relevant information is very well documented on <a href="https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html">Amazon’s website</a>.</p>
<p>Resources can be added to VPCs, and subnets as long as those have available IPs. Running out of addresses will make it impossible to launch new instances. This will impact services that automatically scale, like lambdas.</p>
<p>There are many opinions on how to pick the right size, and I doubt mine has any real value. Instead, I will show how to monitor available IPs.</p>
<h2 id="code">Code</h2>
<p>To avoid dealing with a full server, the application is best as an AWS Lambda. Any runtime with a quick cold start should work. I picked JavaScript.</p>
<p>The <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/EC2.html#describeSubnets-property"><code class="language-plaintext highlighter-rouge">describeSubnets</code></a> function, on the EC2 object, return subnets.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">aws-sdk</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">EC2</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">EC2</span><span class="p">();</span>
<span class="nx">EC2</span><span class="p">.</span><span class="nx">describeSubnets</span><span class="p">({},</span> <span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">err</span><span class="p">.</span><span class="nx">stack</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">data</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>The result, <code class="language-plaintext highlighter-rouge">data</code>, contains only a single page. Using the <code class="language-plaintext highlighter-rouge">NextToken</code> parameter is a solution, but the <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#eachPage-property"><code class="language-plaintext highlighter-rouge">eachPage</code></a>, and the <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#eachItem-property"><code class="language-plaintext highlighter-rouge">eachItem</code></a> function is easier.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">aws-sdk</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">EC2</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">EC2</span><span class="p">();</span>
<span class="nx">EC2</span><span class="p">.</span><span class="nx">describeSubnets</span><span class="p">().</span><span class="nx">eachItem</span><span class="p">((</span><span class="nx">err</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">err</span><span class="p">.</span><span class="nx">stack</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">data</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>AWS requests have another limitation. <code class="language-plaintext highlighter-rouge">describeSubnets</code> only returns subnets for a given region. The one in the environment, or an explicit one when creating the EC2 object.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">aws-sdk</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">EC2</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">EC2</span><span class="p">({</span>
<span class="na">region</span><span class="p">:</span> <span class="dl">"</span><span class="s2">us-east-1</span><span class="dl">"</span>
<span class="p">});</span>
</code></pre></div></div>
<p>To retrieve all subnets, the application must call the function with each region.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">aws-sdk</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">forEachSubnet</span> <span class="o">=</span> <span class="p">(</span><span class="nx">region</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">EC2</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">EC2</span><span class="p">({</span>
<span class="na">region</span><span class="p">:</span> <span class="nx">region</span>
<span class="p">});</span>
<span class="nx">EC2</span><span class="p">.</span><span class="nx">describeSubnets</span><span class="p">().</span><span class="nx">eachItem</span><span class="p">((</span><span class="nx">err</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">err</span><span class="p">.</span><span class="nx">stack</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">data</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">callback</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">};</span>
<span class="nx">forEachSubnet</span><span class="p">(</span><span class="dl">"</span><span class="s2">us-east-1</span><span class="dl">"</span><span class="p">,</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">);</span>
<span class="nx">forEachSubnet</span><span class="p">(</span><span class="dl">"</span><span class="s2">us-east-2</span><span class="dl">"</span><span class="p">,</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">);</span>
</code></pre></div></div>
<p>Hard coding a list of regions isn’t a viable solution. <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/EC2.html#describeRegions-property"><code class="language-plaintext highlighter-rouge">describeRegions</code></a> is preferable. It retrieves every available region on AWS.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">aws-sdk</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">EC2</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">EC2</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">forEachRegion</span> <span class="o">=</span> <span class="nx">callback</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">EC2</span><span class="p">.</span><span class="nx">describeRegions</span><span class="p">().</span><span class="nx">eachItem</span><span class="p">((</span><span class="nx">err</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">err</span><span class="p">.</span><span class="nx">stack</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">data</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">callback</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">};</span>
<span class="nx">forEachRegion</span><span class="p">(</span><span class="nx">region</span> <span class="o">=></span>
<span class="nx">forEachSubnet</span><span class="p">(</span><span class="nx">region</span><span class="p">.</span><span class="nx">RegionName</span><span class="p">,</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">)</span>
<span class="p">);</span>
</code></pre></div></div>
<p>The subnets objects have useful information to extract. Identifiers and the <code class="language-plaintext highlighter-rouge">AvailableIpAddressCount</code> attribute are obvious choices. The total amount of IP addresses isn’t available on the object, but via the <a href="https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing"><code class="language-plaintext highlighter-rouge">CidrBlock</code></a>. Instead of reinventing the wheel, the <a href="https://www.npmjs.com/package/ip-cidr">ip-cidr package</a> offers a quick solution.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">IPCIDR</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">ip-cidr</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">ipAddressCount</span> <span class="o">=</span> <span class="nx">cidrStr</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">cidr</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">IPCIDR</span><span class="p">(</span><span class="nx">cidrStr</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">cidr</span><span class="p">.</span><span class="nx">isValid</span><span class="p">()</span> <span class="p">?</span> <span class="nx">cidr</span><span class="p">.</span><span class="nx">toArray</span><span class="p">().</span><span class="nx">length</span> <span class="p">:</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div></div>
<p>Once gathered, the application can push the data to AWS CloudWatch for alarms.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">CloudWatch</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">./aws/cloudwatch</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">Namespace</span><span class="p">:</span> <span class="dl">"</span><span class="s2">subnet-ip-availability</span><span class="dl">"</span><span class="p">,</span>
<span class="na">MetricData</span><span class="p">:</span> <span class="p">...</span>
<span class="p">};</span>
<span class="nx">CloudWatch</span><span class="p">.</span><span class="nx">putMetricData</span><span class="p">(</span><span class="nx">params</span><span class="p">,</span> <span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span> <span class="o">!==</span> <span class="kc">null</span><span class="p">)</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">err</span><span class="p">.</span><span class="nx">stack</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>With the code finished, it needs to be deployed to AWS.</p>
<h2 id="infrastructure">Infrastructure</h2>
<p>AWS has three official ways to create resources: the console, the <a href="https://aws.amazon.com/cli/">command line interface</a>, and CloudFormation. The last option takes a bit of time, but guaranties the infrastructure to be the same each time.</p>
<p>The main resource is the <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html">AWS Lambda function</a>. It requires a role.</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">AWSTemplateFormatVersion</span><span class="pi">:</span> <span class="s2">"</span><span class="s">2010-09-09"</span>
<span class="na">Description</span><span class="pi">:</span> <span class="s">AWS Lambda IP Availability</span>
<span class="na">Parameters</span><span class="pi">:</span>
<span class="na">Name</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">String</span>
<span class="na">Default</span><span class="pi">:</span> <span class="s2">"</span><span class="s">aws-lambda-ip-availability"</span>
<span class="na">Resources</span><span class="pi">:</span>
<span class="na">Role</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::IAM::Role</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">RoleName</span><span class="pi">:</span> <span class="kt">!Ref</span> <span class="s">Name</span>
<span class="na">AssumeRolePolicyDocument</span><span class="pi">:</span>
<span class="na">Version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">2012-10-17"</span>
<span class="na">Statement</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">Effect</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Allow"</span>
<span class="na">Action</span><span class="pi">:</span> <span class="s2">"</span><span class="s">sts:AssumeRole"</span>
<span class="na">Principal</span><span class="pi">:</span>
<span class="na">Service</span><span class="pi">:</span> <span class="s2">"</span><span class="s">lambda.amazonaws.com"</span>
<span class="na">Function</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::Lambda::Function</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">FunctionName</span><span class="pi">:</span> <span class="kt">!Ref</span> <span class="s">Name</span>
<span class="na">Role</span><span class="pi">:</span> <span class="kt">!GetAtt</span> <span class="s">Role.Arn</span>
<span class="na">Runtime</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nodejs8.10"</span> <span class="c1"># nodejs10.x doesn't support zip file</span>
<span class="na">Handler</span><span class="pi">:</span> <span class="s2">"</span><span class="s">index.handler"</span>
<span class="na">Code</span><span class="pi">:</span>
<span class="na">ZipFile</span><span class="pi">:</span> <span class="s2">"</span><span class="s">//"</span>
</code></pre></div></div>
<p>AWS Lambda writes to CloudWatch Logs. Their role needs the permissions to interact with Log Groups and Log Streams.</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">LogGroup</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::Logs::LogGroup</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">RetentionInDays</span><span class="pi">:</span> <span class="m">7</span>
<span class="na">LogGroupName</span><span class="pi">:</span> <span class="kt">!Join</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">"</span><span class="pi">,</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">/aws/lambda/"</span><span class="pi">,</span> <span class="kt">!Ref</span> <span class="nv">Name</span> <span class="pi">]</span> <span class="pi">]</span>
<span class="na">RoleCloudWatchLog</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::IAM::Policy</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">PolicyName</span><span class="pi">:</span> <span class="kt">!Join</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">"</span><span class="pi">,</span> <span class="pi">[</span> <span class="kt">!Ref</span> <span class="nv">Name</span><span class="pi">,</span> <span class="s2">"</span><span class="s">-cloudwatch-log"</span> <span class="pi">]</span> <span class="pi">]</span>
<span class="na">PolicyDocument</span><span class="pi">:</span>
<span class="na">Version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">2012-10-17"</span>
<span class="na">Statement</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">Effect</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Allow"</span>
<span class="na">Action</span><span class="pi">:</span> <span class="s2">"</span><span class="s">logs:CreateLogGroup"</span>
<span class="na">Resource</span><span class="pi">:</span> <span class="kt">!Join</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">"</span><span class="pi">,</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">arn:aws:logs:"</span><span class="pi">,</span> <span class="kt">!Ref</span> <span class="s2">"</span><span class="s">AWS::Region"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">:"</span><span class="pi">,</span> <span class="kt">!Ref</span> <span class="s2">"</span><span class="s">AWS::AccountId"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">:log-group:"</span><span class="pi">,</span> <span class="kt">!Ref</span> <span class="nv">LogGroup</span> <span class="pi">]</span> <span class="pi">]</span>
<span class="pi">-</span>
<span class="na">Effect</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Allow"</span>
<span class="na">Action</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">logs:CreateLogStream"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">logs:PutLogEvents"</span>
<span class="na">Resource</span><span class="pi">:</span> <span class="kt">!GetAtt</span> <span class="s">LogGroup.Arn</span>
<span class="na">Roles</span><span class="pi">:</span>
<span class="pi">-</span> <span class="kt">!Ref</span> <span class="s">Role</span>
</code></pre></div></div>
<p>The application also interacts with EC2 and CloudWatch.</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">RoleEc2</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::IAM::Policy</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">PolicyName</span><span class="pi">:</span> <span class="kt">!Join</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">"</span><span class="pi">,</span> <span class="pi">[</span> <span class="kt">!Ref</span> <span class="nv">Name</span><span class="pi">,</span> <span class="s2">"</span><span class="s">-ec2"</span> <span class="pi">]</span> <span class="pi">]</span>
<span class="na">PolicyDocument</span><span class="pi">:</span>
<span class="na">Version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">2012-10-17"</span>
<span class="na">Statement</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">Effect</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Allow"</span>
<span class="na">Action</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">ec2:DescribeRegions"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">ec2:DescribeSubnets"</span>
<span class="na">Resource</span><span class="pi">:</span> <span class="s2">"</span><span class="s">*"</span>
<span class="na">Roles</span><span class="pi">:</span>
<span class="pi">-</span> <span class="kt">!Ref</span> <span class="s">Role</span>
<span class="na">RoleCloudWatchMetric</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::IAM::Policy</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">PolicyName</span><span class="pi">:</span> <span class="kt">!Join</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">"</span><span class="pi">,</span> <span class="pi">[</span> <span class="kt">!Ref</span> <span class="nv">Name</span><span class="pi">,</span> <span class="s2">"</span><span class="s">-cloudwatch-metric"</span> <span class="pi">]</span> <span class="pi">]</span>
<span class="na">PolicyDocument</span><span class="pi">:</span>
<span class="na">Version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">2012-10-17"</span>
<span class="na">Statement</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">Effect</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Allow"</span>
<span class="na">Action</span><span class="pi">:</span> <span class="s2">"</span><span class="s">cloudwatch:PutMetricData"</span>
<span class="na">Resource</span><span class="pi">:</span> <span class="s2">"</span><span class="s">*"</span>
<span class="na">Roles</span><span class="pi">:</span>
<span class="pi">-</span> <span class="kt">!Ref</span> <span class="s">Role</span>
</code></pre></div></div>
<p>The template can also hold a periodic event to trigger the application.</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">Event</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::Events::Rule</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">Name</span><span class="pi">:</span> <span class="kt">!Ref</span> <span class="s">Name</span>
<span class="na">ScheduleExpression</span><span class="pi">:</span> <span class="s2">"</span><span class="s">rate(1</span><span class="nv"> </span><span class="s">hour)"</span>
<span class="na">Targets</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">Id</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Target-1"</span>
<span class="na">Arn</span><span class="pi">:</span> <span class="kt">!GetAtt</span> <span class="s">Function.Arn</span>
<span class="na">EventPermission</span><span class="pi">:</span>
<span class="na">Type</span><span class="pi">:</span> <span class="s">AWS::Lambda::Permission</span>
<span class="na">Properties</span><span class="pi">:</span>
<span class="na">Principal</span><span class="pi">:</span> <span class="s2">"</span><span class="s">events.amazonaws.com"</span>
<span class="na">Action</span><span class="pi">:</span> <span class="s2">"</span><span class="s">lambda:InvokeFunction"</span>
<span class="na">FunctionName</span><span class="pi">:</span> <span class="kt">!Ref</span> <span class="s">Function</span>
<span class="na">SourceArn</span><span class="pi">:</span> <span class="kt">!GetAtt</span> <span class="s">Event.Arn</span>
</code></pre></div></div>
<p>Every execution puts metrics on CloudWatch to trigger alarms.</p>
<h2 id="alarms">Alarms</h2>
<p><a href="https://console.aws.amazon.com/cloudwatch/home?region=us-east-1">AWS CloudWatch Alarm</a> monitor metrics. When certain conditions are met, a notification is sent to an SNS topic. All subscribers will receive it too. Those can be email addresses, phone numbers, HTTP endpoints, and more.</p>
<p>The metric should either be the amount of available IPs, or the percentage of remaining ones.</p>
<p>When this metric is below an acceptable threshold, the alarm should message the SNS topic. This will draw attention to the subnet to add another or replace it by a larger one.</p>
<p>Adding alarms for every subnet is quite a repetitive task. Using the CLI, or building a small application can make the process less of a chore. Something for the next post … maybe.</p>
<hr />
<p>Running out of available IPs is a silly move that can happen to anyone. The errors are rarely displayed in the appropriate service making it very hard to debug. This little application, <a href="https://github.com/plippe/aws-lambda-ip-availability">available on GitHub</a>, will hopefully avoid a few sleepless nights.</p>Philippe VinchonRunning virtual private cloud, or VPC, on AWS is common, and even required in some cases.