After writing a few bash scripts for various stages in GitLab CI/CD pipelines, I stumbled upon a great Node.js library
from Google, called <code>zx</code> (<a href="https://github.com/google/zx">link</a>).
It wraps Node.js's <code>child_process</code> modules, which deals with subprocesses and gives access to
the <code>stdout</code> (<a href="https://nodejs.org/api/child_process.html#child-process">link</a>).
The only catch was that the library is only exported as an ECMAScript module. At the same time the intention was to
use <code>zx</code> in a TypeScript/Node.js project, that generates <code>commonjs</code> modules that don't go along with modules, like <code>zx</code>.
<h2>ECMAScript Modules</h2>
You can read more about the nature of ECMAScript modules in a great blog article on
Formidable.com (<a href="https://formidable.com/blog/2021/node-esm-and-exports/">link</a>) and one provided by
Mozilla (<a href="https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/">link</a>):
<blockquote>
ES modules bring an official, standardized module system to JavaScript. It took a while to get here, though
nearly 10 years of standardization work.
</blockquote>
Parts of the existing project was also dependent on <code>ts-node</code> (<a href="https://github.com/TypeStrong/ts-node">link</a>) for
executing TypeScript files &quot;on the fly&quot; and <code>ts-node-dev</code> (<a href="https://github.com/wclr/ts-node-dev">link</a>) for &quot;live
TypeScript coding&quot; with automatic, efficient restarts.
As soon as I started making the changes and reading the <code>ts-node</code> docs, I noticed the following
statement (<a href="https://typestrong.org/ts-node/docs/recipes/watching-and-restarting">link</a>):
<blockquote>
There's also ts-node-dev, a modified version of node-dev using ts-node for compilation that will restart the process
on file change. Note that ts-node-dev is incompatible with our native ESM loader.
</blockquote>
<h2>Minimal Setup</h2>
Okay, fun. Before hitting any more road blocks by turning too many screws at the same time, I decided to spin up a new
TypeScript project and focus on the minimal set of changes necessary to get things going.
<pre class="language-bash"><code class="language-bash">npm init --yes
npm install -D typescript
npm install -D ts-node
npm install -D @types/node
</code></pre>
That's a great start. What's left to do, is to work on a few configurations. So let's create a TypeScript configuration
file first:
<pre class="language-bash"><code class="language-bash">touch tsconfig.json
</code></pre>
There is a fantastic GitHub project <code>tsconfig/bases</code> that provides some sane defaults, that can be used as a foundation:
<pre class="language-bash"><code class="language-bash">npm install --save-dev @tsconfig/node16-strictest
</code></pre>
<h2>TypeScript and Node Configuration</h2>
Inside our <code>tsconfig.json</code>, we can now make the following changes:
<pre class="language-json"><code class="language-json">{
 "extends": "@tsconfig/node16-strictest/tsconfig.json",
 "compilerOptions": {
 "module": "ESNext",
 "moduleResolution": "node",
 "outDir": "dist"
 }
}
</code></pre>
Initially the aforementioned default configuration is extended. Next, <code>module</code> specifies what module code is
generated. <code>ESNext</code> refers to the common <code>import/export</code> expressions instead of transforming those into <code>require()</code>
ones.
<code>moduleResolution</code> describes how TypeScript looks up a file from a given module specifier. It basically tells TypeScript
how to find this module, referenced in a line like <code>import { ... } from 'src/func'</code>.
As it defers to <code>node</code> for the module resolution, we need to add one additional line in our <code>package.json</code> file:
<pre class="language-json"><code class="language-json">{
 ...
 "type": "module",
 ...
}
</code></pre>
This property value tells Node to use its ECMAScript modules loader to load the referenced
modules (<a href="https://nodejs.org/api/esm.html#enabling">link</a>).
<h2>First Build</h2>
There is nothing more to do to transpile TypeScript files and run the generated JavaScript files with node. You could
create a new file in <code>src/index.ts</code> and add a function in there:
<pre class="language-typescript"><code class="language-typescript">void async function () {
 console.log('Et voila!')
}()
</code></pre>
Running <code>tsc</code> would lead to the generation of a build folder (in this case <code>dist</code>) that contains the generated files.
<h2>Executing TypeScript Files</h2>
Sometimes it's not necessary to transpile TypeScript into JavaScript. It is sufficient to execute the files directly,
for example to execute them within a CI/CD pipeline. In this case it's handy to set up <code>ts-node</code> as well, which is very
well explained in their docs (<a href="https://typestrong.org/ts-node/docs/imports#native-ecmascript-modules">link</a>).
The following lines need to be added to <code>tsconfig.json</code>:
<pre class="language-json"><code class="language-json">{
 ...
 "ts-node": {
 "esm": true
 }
}
</code></pre>
Executing a TypeScript file is as easy as executing the following file:
<pre class="language-bash"><code class="language-bash">ts-node src/index.ts
</code></pre>
There is one (huge) downside mentioned in the docs considering the support for ESM in production:
<blockquote>
Node's ESM loader hooks are experimental and subject to change. ts-node's ESM support is as stable as possible, but
it relies on APIs which node can and will break in new versions of node. Thus it is not recommended for production.
</blockquote>
For now, I assume that as long as you hold on to your versions and are aware that upgrades to newer <code>ts-node</code> and
<code>node.js</code> version might be a bit slower due to the lack of backward compatibility of those instable APIs.
<h2>Speed Things Up with SWC</h2>
There are few more options you could set in your <code>tsconfig.json</code> in order to speed things up
with <code>ts-node</code> (<a href="https://typestrong.org/ts-node/docs/performance#skip-typechecking">link</a>):
<pre class="language-json"><code class="language-json">{
 "ts-node": {
 "esm": true,
 "files": true,
 "transpileOnly": true,
 "swc": true
 }
}
</code></pre>
The biggest performance gain comes from using the super fast rust based transpiler called <code>swc</code> and skipping type
checks.
<h2>Module Resolutions</h2>
Remember the module resolutions mentioned earlier? There is another catch, when you create a second TypeScript file,
such as <code>moduleA.ts</code> and try to import it in <code>index.ts</code>:
<pre class="language-typescript"><code class="language-typescript">import {func2} from './moduleA'

void async function () {
 console.log(func2())
}()
</code></pre>
This leads to the following error message complaining that it can't find <code>moduleA</code>:
<pre class="language-bash"><code class="language-bash">throw new ERR_MODULE_NOT_FOUND(...)

CustomError: Cannot find module '.../src/moduleA' imported from ...src/index.ts
</code></pre>
After some more digging and confusion, why it would not automatically find <code>moduleA.ts</code> when referencing it
as <code>./moduleA</code>, I found the answer in the Node.js docs as
well (<a href="https://nodejs.org/dist/latest-v18.x/docs/api/esm.html#mandatory-file-extensions">link</a>):
<blockquote>
A file extension must be provided when using the import keyword to resolve relative or absolute specifiers. Directory indexes (e.g. './startup/index.js') must also be fully specified.
This behavior matches how import behaves in browser environments, assuming a typically configured server.
</blockquote>
That means it's absolutely necessary to add the <code>.ts</code> extension unless the experimental usage of automatic file
extension resolutions is used
(<a href="https://nodejs.org/dist/latest-v18.x/docs/api/esm.html#customizing-esm-specifier-resolution-algorithm">link</a>):
In <code>tsconfig.json</code> the following file needs to be added:
<pre class="language-json"><code class="language-json">{
 ...
 "tsconfig": {
 ...
 "experimentalSpecifierResolution": "node",
 ...
 }
}
</code></pre>
And once the files are transpiled, the files can only be executed using <code>node</code> with a flag:
<pre class="language-bash"><code class="language-bash">node --experimental-specifier-resolution=node dist/index.js
</code></pre>
The Node.js documentation warns against relying on the usage of this flag, so it's far from perfect:
<blockquote>
Do not rely on this flag. We plan to remove it once the Loaders API has advanced to the point that equivalent functionality can be achieved via custom loaders.
The current specifier resolution does not support all default behavior of the CommonJS loader. One of the behavior differences is automatic resolution of file extensions and the ability to import directories that have an index file.
</blockquote>
<h2>Conclusion</h2>
There still exists a high dynamic in the use and implementation of ECMAScript modules. This is true for both Node.js and
TypeScript executors like <code>ts-node</code>.
Accordingly, an increased maintenance effort is to be expected in a production system when upgrading versions.

ECMAScript Modules With Typescript

A few good arguments for adopting an asynchronous messaging pattern in a microservice environment include high scalability,
loose coupling and better use of resources. Instead of continuously polling the source system, the flow of communication is inverted. Messages are produced by the source system in order to inform consumers of noteworthy events. Those events are expressed in messages that are distributed by a message broker. The design of those messages has a direct impact on the implementation of the producing and consuming services.
<h2>Message Design</h2>
Events are supposed to inform consumers that something has happened. In frontend applications that could either apply to an application state change triggered by an action or to a context change, an event captured by the browser, like a page scroll, cursor movements and there like. Without loss of generality, in a classic CRUD based backend application it usually relates to a state change: A record was inserted, updated or deleted.
<h3>Operations on Entities</h3>
A simple representation of an event is the full state of a data item and a fixed set of operations that can be applied to this state. This is quite similar to the application of the REST pattern, where a <code>POST</code> (creation) and <code>PUT</code> (update) HTTP request include the full payload of the entity to be inserted or updated. In order to inform the consumer about the nature of the event, metadata could be added to the event.
For a todo that was just added to a list of todos, the message representing such an event could look like this:
<pre class="language-json"><code class="language-json">{
 "metadata": {
 "action": "INSERT"
 },
 "data": {
 "id": 1,
 "text": "My first todo.",
 "completed": false
 }
}
</code></pre>
Let's imagine the owner of this todo has completed the task and an <code>UPDATE</code> message is published. It is structured in the same way, only now the <code>action</code> property is assigned to <code>UPDATE</code> and the <code>completed</code> property is set to <code>true</code>. For a consumer it is not clear, what triggered the event and what in particular changed. It is only obvious, that something has changed, and it is up to the consumer to find out what it was.
In some scenarios this could still be sufficient though: If it is not necessary to react to events but rather store the latest state for the purpose of extending the service's dataset, it should be fine to include the full state in each message. Choosing this approach also reduces the dependency between each event sine every event self-contained as it includes all the information to represent a data item's current state.
<h3>Occurrences of Actions</h3>
Another way to tackle the message design is to think of events as occurrences of actions in a well-defined process. Every action leads to state change. Therefore, each message should describe the action that occurred and the properties of the state that changed as a consequence.
A completed todo item could be represented for example like this:
<pre class="language-json"><code class="language-json">{
 "event": "TodoCompleted",
 "id": 1
}
</code></pre>
The consumer is now able to easily understand what happened (<code>&quot;event&quot;: &quot;TodoCompleted&quot;</code>). The reaction to that event is entirely up to the consumer. It could mean updating the todo's state by setting the <code>complete</code> property to <code>false</code>.
In the end a series of events leads to the current state of an entity. This is called &quot;event sourcing&quot;. Although the messages are much more succinct and carry a stronger semantic meaning as they represent an event very specifically, there are extra costs on the implementation side. It's up to the consumer to interpret those events and to react to them appropriately. There could be a multitude of events that are emitted, which makes documentation essential.
<h3>Order</h3>
Ideally, events are self-contained in a sense that they do not have any other dependencies towards other events and can appear in any particular order. This reduces complexity both for the message queue, which might otherwise be required to order messages by a certain criterion, and the consumer, that otherwise needs to make additional queries and decision based on the order of events.
If the text of a task has changed multiple times (e.g. <code>ToDoTextChanged</code>) and the events are not ordered, the consumer has no way of determining the most recent text of the todo item.
This can be solved by adding a criterion by which the consumer is able to order the sequence or decide on an appropriate action. If the service has already seen at least one event and the following one has a lower sequence number, it can simply choose to ignore this event for example. But this also means the service needs to query the latest sequence number or store it in memory, make the comparison, which introduces additional overhead.
<h2>Other Event Types</h2>
Events are pretty flexible in what they actually express. The word &quot;message&quot; was chosen deliberately, since it carries the notion of informing the consumers about an event. As soon as the message leaves the sender, the sender's job is done, and it has no idea how the consumers of the message will react to it. This is exactly what manifests the loose coupling between sender and consumer and enables the scaling as the sender can carry on with its duties, while the message broker distributes the message amongst the consumers.
There are other patterns, like the <a href="https://martinfowler.com/bliki/CQRS.html">Command Query Responsibility Segregation (CQRS)</a>, pattern, that describes asynchronous communication based on &quot;query&quot; and &quot;command&quot; events. In contrast to plain messages, the sender sends a message with the clear intent to invoke a certain action or process on the receiver's side. Furthermore, the result of the invocation is even sent back to the original sender. In this case asynchronicity is they key selling point, but loose coupling only to a certain extent - if at all.
So the question is, what fits the underlying business process and what technological requirements are in place. Based on the answer, the implementation of an event-driven architecture can be designed. Of course there are also enough cases, that don't fit the asynchronous messaging approach at all.

Message Design In An Event Driven Architecture

I've been working on an enterprise reporting tool for the past year.
The frontend of that tool consists only of a couple of views and one of them
is a relatively complex form, which is also the heart of this reporting tool.
<h2>Challenges</h2>
It consists of a lot of features that make the implementation of a form quite
challenging. There are dropdown fields filled with items dynamically loaded from
an API endpoint (1).
Other dropdowns show a list of items based on the selection of a dependent field (2).
The money input fields format the number based on the chosen locale while keeping
an unformatted, pure JavaScript number in the state. Other money input fields
depend on one or multiple money input fields and are sometimes calculated
automatically (3,5). To make things harder, some calculations are bidirectional
between multiple fields.
Imagine a price value and a commission percentage. Multiplying those values equals
the value of the commission input. Should the change of the commission now change
the percentage or the price value?
And to make things even more exciting, complete parts of the form are exchanged
based on other fields (4).
<img 
 src="../images/articles/formik-in-complex-forms/form.png"
 srcset="../images/articles/formik-in-complex-forms/form.png,
 ../images/articles/formik-in-complex-forms/form@2x.png 2x"
 alt="Simplified mockup of the complex form"
/>
<h2>Formik</h2>
It makes sense to choose a library to handle all the internals of form state handling,
validation and so on not to reinvent the wheel. Formik is such a library: It has earned
more than 28.5k stars on GitHub (<a href="https://github.com/formium/formik">link</a>)
and is even mentioned briefly in the React documentation (<a href="https://reactjs.org/docs/forms.html#fully-fledged-solutions">link</a>).
Formik comes with a couple of appealing features: Support for a custom validation
mechanism, provides hooks to read from and update the state as well as a couple
of standard handlers for <code>onChange</code> and <code>onBlur</code>. It also has a component to deal
with lists and offers functions to deal with them accordingly (e.g. adding and
removing items).
With more than twenty fields with such special behaviors as described previously,
each of which reacts to every key press, a form library is challenged not only in
terms of its functionality, but also in terms of performance.
<h2>Performance</h2>
And indeed an input lag was noticeable which led to the discovery of Formik's
<code>FastField</code> component (<a href="https://formik.org/docs/api/fastfield">link</a>):
<blockquote>
<code>&lt;FastField /&gt;</code> is an optimized version of <code>&lt;Field /&gt;</code> meant to be used on large forms
(~30+ fields) or when a field has very expensive validation requirements.
</blockquote>
The problem is that every field update causes the whole form state to be rewritten.
This on the other hand triggers every field that is either rendered with the
<code>&lt;Field /&gt;</code> component or uses the <code>useField()</code> hook.
This is calling for trouble and the sheer co-existence of <code>&lt;FastField /&gt;</code> raises
the question, why the other components are not optimized for performance from the
beginning on.
One a half years ago a GitHub issue was opened to raise awareness to the performance
issues that come with the <code>useField()</code> hook, but it's not resolved. According to a
GitHub user commenting in this thread (<a href="https://github.com/formium/formik/issues/2268">link</a>)
Formik's architecture makes it nearly impossible to implement an efficient solution.
Instead he proposes to rely on <code>React.memo</code> to avoid unnecessary rerenders.
Unfortunately that's close to the solution we were forced to apply in order to
prevent those costly rerenders: Intead of wrapping the <code>useField()</code> hook, which is
smart, we decided to wrap our custom input field components, like the money input
field and the dropdowns for example.
<h2>Conclusion</h2>
Formik offers a clean documentation, a great set of hooks and components to get
the job done, and is a sufficient solution for smaller forms. Due to the serious
performance bottlenecks in complex forms and the lack of improvement over the
course of the past one and a half years, a choice in favor of Formik should
be considered carefully.
React Final Form (<a href="https://final-form.org/react">link</a>) is another library that offers &quot;high performance subscription-based form state management&quot;:
<blockquote>
For small forms, redrawing your entire form on every keypress is no problem. But when your form grows, performance can degrade.
</blockquote>
<blockquote>
No other form library allows such fine tuning to manage exactly which form elements get notified of form state changes.
</blockquote>
Its API is quite similar to Formik in a lot of components, and therefore it's relatively
easy to switch to this solution. There is even a migration guide available (<a href="https://final-form.org/docs/react-final-form/migration/formik">link</a>).
Additional features, like decorators (<a href="https://final-form.org/docs/final-form/types/Decorator">link</a>), only add to its
value and reduce the amount of code that needs to implemented manually otherwise.