<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>statemachine &#8211; Software, Fitness, and Gaming &#8211; Jesse Warden</title>
	<atom:link href="https://jessewarden.com/tag/statemachine/feed" rel="self" type="application/rss+xml" />
	<link>https://jessewarden.com</link>
	<description>Software &#124; Fitness &#124; Gaming</description>
	<lastBuildDate>Tue, 07 Apr 2020 04:21:21 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	

<image>
	<url>https://jessewarden.com/wp-content/uploads/2016/08/cropped-Lambda2-32x32.png</url>
	<title>statemachine &#8211; Software, Fitness, and Gaming &#8211; Jesse Warden</title>
	<link>https://jessewarden.com</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Handling Noop Infrastructure in AWS Step Functions</title>
		<link>https://jessewarden.com/2020/04/handling-noop-infrastructure-in-aws-step-functions.html</link>
		
		<dc:creator><![CDATA[JesterXL]]></dc:creator>
		<pubDate>Tue, 07 Apr 2020 04:21:21 +0000</pubDate>
				<category><![CDATA[JavaScript]]></category>
		<category><![CDATA[aws]]></category>
		<category><![CDATA[functionalprogramming]]></category>
		<category><![CDATA[statemachine]]></category>
		<category><![CDATA[states]]></category>
		<category><![CDATA[stepfunction]]></category>
		<guid isPermaLink="false">https://jessewarden.com/?p=5921</guid>

					<description><![CDATA[Spent a month on this and I think I figured out how to get JSON to &#8220;roll over&#8221; a Step Function step that is a noop (no operation: function that doesn&#8217;t return a value), like say AWS Batch. Noop&#8217;s (no operation) are steps that basically are full of side effects, but really don&#8217;t have a [&#8230;]]]></description>
										<content:encoded><![CDATA[
<p>Spent a month on this and I think I figured out how to get JSON to &#8220;roll over&#8221; a <a href="https://aws.amazon.com/step-functions/">Step Function</a> step that is a noop (no operation: function that doesn&#8217;t return a value), like say <a href="https://aws.amazon.com/batch/">AWS Batch</a>. Noop&#8217;s (no operation) are steps that basically are full of side effects, but really don&#8217;t have a useful return value beyond &#8220;SUCCESS&#8221;.</p>



<span id="more-5921"></span>



<p>If you code Batch in <a href="https://golang.org/">Go</a> for example, the main method has it right there in the return value: void. Meaning… there <em>is</em> no return value.</p>



<p>How, then, do you use something like a noop in your mostly Functional Programming style Step Function that assumes you&#8217;re composing infrastructure like you&#8217;d compose functions? Basically tap. If you&#8217;re not familiar, tap is a quick way to debug pure function chains.</p>



<p>If you&#8217;ve ever heard of the identity function, it basically gives you back what you give it, like this in Python:</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript">def identity(oh_yeah):
  <span class="hljs-keyword">return</span> oh_yeah</code></span></pre>


<p>Or this in JavaScript:</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript"><span class="hljs-keyword">const</span> identity = <span class="hljs-function"><span class="hljs-params">ohYeah</span> =&gt;</span>
  ohYeah</code></span></pre>


<p>That seems like a ridiculous function, no doubt. But watch what happens when you combine it with composed functions, say, parsing some JSON.</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript">def parse(str):
  <span class="hljs-keyword">return</span> json_parse(str) \
  &gt;&gt; parse_people \
  &gt;&gt; filter_humans \
  &gt;&gt; format_names</code></span></pre>


<p>Parse composes 4 functions together. If they&#8217;re pure, how do you &#8220;see&#8221; inside that pipeline? You &#8220;tap&#8221; into it. We can take our identity function, and convert to a tap function like this:</p>


<pre class="wp-block-code"><span><code class="hljs language-php">def tap(oh_yeah):
  <span class="hljs-keyword">print</span>(<span class="hljs-string">"oh_yeah:"</span>, oh_yeah)
  <span class="hljs-keyword">return</span> oh_yeah</code></span></pre>


<p>Then we can &#8220;tap into&#8221; the pipeline.</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript">def parse(str):
  <span class="hljs-keyword">return</span> json_parse(str) \
  &gt;&gt; tap \
  &gt;&gt; parse_people \
  &gt;&gt; tap \
  &gt;&gt; filter_humans \
  &gt;&gt; tap \
  &gt;&gt; format_names</code></span></pre>


<p>Sick, ya? Now the normal way to do that in Step Functions is via a <a href="https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-pass-state.html">Pass state</a>:</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript"><span class="hljs-string">"Tap"</span>: {
  <span class="hljs-string">"Type"</span>: <span class="hljs-string">"Pass"</span>,
  <span class="hljs-string">"Next"</span>: <span class="hljs-string">"NextStep"</span>
}</code></span></pre>


<p>You can then see the output in the Step Function console, or tweak the inputs/outputs if you wish.</p>



<p>… but that&#8217;s for Lambdas, Step Functions, SQS, and SNS steps that not only return a result like a pure function, but do so using your JSON. In JavaScript, that&#8217;s pretty easy to merge properties:</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript"><span class="hljs-keyword">const</span> addYoSelf = <span class="hljs-function"><span class="hljs-params">json</span> =&gt;</span>
  ({ …json, <span class="hljs-attr">name</span>: <span class="hljs-string">"Jesse was here"</span> })</code></span></pre>


<p>Batch, and even Step Functions, don&#8217;t always work like this. You typically have to define <a href="https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html">Parameters</a> manually depending upon the input… meaning you lose your JSON tree. <img src="https://s.w.org/images/core/emoji/17.0.2/72x72/1f622.png" alt="😢" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>



<p>… UNLESS it&#8217;s just 1 branch. You can create siblings that are collected in a single spot using the <a href="https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-parallel-state.html">Parallel task</a>.</p>



<p>If you think <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all">Promise.all</a> like in JavaScript or <a href="https://docs.python.org/3/library/asyncio-task.html#asyncio.gather">gather</a> in Python, multiple results, but a single, Array filled result. This allows you to keep context at least in 1 spot using a Pass, and you don&#8217;t care about the rest.</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript">a = stuff
identity = <span class="hljs-function"><span class="hljs-params">stuff</span> =&gt;</span> stuff
addSelf = <span class="hljs-function"><span class="hljs-params">stuff</span> =&gt;</span> ({… stuff, <span class="hljs-attr">name</span>: <span class="hljs-string">'Jesse'</span>})
destroy = <span class="hljs-function"><span class="hljs-params">stuff</span> =&gt;</span> <span class="hljs-literal">undefined</span>
end = <span class="hljs-function"><span class="hljs-params">a</span> =&gt;</span> <span class="hljs-built_in">Promise</span>.all(&#91;identity(a), addSelf(a), destroy(a)])</code></span></pre>


<p>Notice how <code>end</code> will result in an Array like:<br><code>[stuff, stuff + name Jesse, undefined]</code></p>



<p>Undefined is often what we get with noops that return no value. BUT, we need the first part to continue our Step Function with &#8220;context&#8221;. Using a Parallel + Pass, you can keep that context.</p>



<p>Until Batch gets the .waitForTaskToken feature (I mean… ECS has it…), you can wrap her in a Parallel to keep context. Just watch those catches….</p>



<p>tl;dr; JSON use Parallel and in next state go:</p>


<pre class="wp-block-code"><span><code class="hljs language-javascript"><span class="hljs-string">"OutputPath"</span>: <span class="hljs-string">"$&#91;0]"</span></code></span></pre>


<p>BTW, I know you can use <code>ResultPath</code> and just attach to your existing JSON, but AWS Batch is HUGE, and we&#8217;re already getting super close to our JSON size limit with our immense Array size I refuse to read from S3. You could just export and then rip off from a Pass state, but you&#8217;ll get a size exception before that happens sadly. Twice now I&#8217;ve got an exception when it completes the Batch because the JSON is too large, and using this technique has worked every time. </p>



<figure class="wp-block-gallery columns-1 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex"><ul class="blocks-gallery-grid"><li class="blocks-gallery-item"><figure><img fetchpriority="high" decoding="async" width="372" height="317" src="https://jessewarden.com/wp-content/uploads/2020/04/stepfunctions_graph-8.png" alt="" data-id="5922" data-full-url="https://jessewarden.com/wp-content/uploads/2020/04/stepfunctions_graph-8.png" data-link="https://jessewarden.com/?attachment_id=5922" class="wp-image-5922" srcset="https://jessewarden.com/wp-content/uploads/2020/04/stepfunctions_graph-8.png 372w, https://jessewarden.com/wp-content/uploads/2020/04/stepfunctions_graph-8-300x256.png 300w" sizes="(max-width: 372px) 100vw, 372px" /></figure></li></ul></figure>



<p>And Step Function JSON for above (supply your own Lambda):</p>


<pre class="wp-block-code"><span><code class="hljs language-json">{
  <span class="hljs-attr">"Comment"</span>: <span class="hljs-string">"A Hello World example of the Amazon States Language using Pass states"</span>,
  <span class="hljs-attr">"StartAt"</span>: <span class="hljs-string">"Both"</span>,
  <span class="hljs-attr">"States"</span>: {
    <span class="hljs-attr">"Both"</span>: {
      <span class="hljs-attr">"Type"</span>: <span class="hljs-string">"Parallel"</span>,
      <span class="hljs-attr">"Next"</span>: <span class="hljs-string">"Keep JSON"</span>,
      <span class="hljs-attr">"Branches"</span>: &#91;
        {
          <span class="hljs-attr">"StartAt"</span>: <span class="hljs-string">"Good Function Returns Value"</span>,
          <span class="hljs-attr">"States"</span>: {
            <span class="hljs-attr">"Good Function Returns Value"</span>: {
              <span class="hljs-attr">"Type"</span>: <span class="hljs-string">"Task"</span>,
              <span class="hljs-attr">"Resource"</span>: <span class="hljs-string">"arn:aws:lambda:us-east-1:0000000:function:jesse-identity"</span>,
              <span class="hljs-attr">"Parameters"</span>: {
                <span class="hljs-attr">"uno"</span>: <span class="hljs-string">"yup"</span>
              },
              <span class="hljs-attr">"ResultPath"</span>: <span class="hljs-string">"$.datLambdaResult"</span>,
              <span class="hljs-attr">"End"</span>: <span class="hljs-literal">true</span>
            }
          }
        },
        {
          <span class="hljs-attr">"StartAt"</span>: <span class="hljs-string">"Bad Noop"</span>,
          <span class="hljs-attr">"States"</span>: {
            <span class="hljs-attr">"Bad Noop"</span>: {
              <span class="hljs-attr">"Type"</span>: <span class="hljs-string">"Pass"</span>,
              <span class="hljs-attr">"Result"</span>: <span class="hljs-string">"World"</span>,
              <span class="hljs-attr">"End"</span>: <span class="hljs-literal">true</span>
            }
          }
        }
      ]
    },
    <span class="hljs-attr">"Keep JSON"</span>: {
      <span class="hljs-attr">"Type"</span>: <span class="hljs-string">"Pass"</span>,
      <span class="hljs-attr">"OutputPath"</span>: <span class="hljs-string">"$&#91;0]"</span>,
      <span class="hljs-attr">"End"</span>: <span class="hljs-literal">true</span>
    }
  }
}</code></span></pre>]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
