Alex Korablevhttps://www.alexkorablev.com/2022-05-24T10:00:00+05:0085% Python, 15% Java, and 100% BackendWhy I don’t have analytics and comments on my website2022-05-24T10:00:00+05:002022-05-24T10:00:00+05:00akorablevtag:www.alexkorablev.com,2022-05-24:/no-more-analytics.html<p><strong>TLDR;</strong> No more analytics, share buttons and comments on this blog. Now it is GDPR-friendly as much as possible without cookie banners. All discussions are on Twitter now. At least while I can use it - I’m from Russia. All my posts starting with this one will have a special link to twitter below the text.</p>
<p><strong>TLDR;</strong> No more analytics, share buttons and comments on this blog. Now it is GDPR-friendly as much as possible without cookie banners. All discussions are on Twitter now. At least while I can use it - I’m from Russia. All my posts starting with this one will have a special link to twitter below the text.</p>
<p>I’ve decided to delete analytics, comments and share buttons from all my websites. Why? The main reason is that I don’t want to have cookie settings pop-up like almost all websites have. The cookie banner itself is not the worst thing. You have to have a cookie policy. Great, I can use one from a template. There are thousands of them available online free or paid. But can I actually follow that policy if I use third-party services? Probably…</p>
<p>The only possible way to be sure that my blog is GDPR-compliant is to ditch all functions that require cookies. For my websites these functions are:</p>
<ul>
<li>Share buttons</li>
<li>Analytics (Google or Yandex.Metrika)</li>
<li>Comments</li>
</ul>
<p><em>Is anything from this list necessary?</em></p>
<p>Definitely not.</p>
<p><em>Do I use anything from that list to find new topics for posts?</em></p>
<p>No.</p>
<p><em>Do I use them at all?</em></p>
<p>Well… No.</p>
<p>The main goal of all my websites is to be my second brain. Probably, Obsidian, Org-mode, or any other similar system is enough for that. But writing posts that are meant to be available for everyone forces me to organize my thoughts carefully. If I write a post on any topic, I do extra research. I try to find a set of the best examples, links, and so on. And the best thing is that I clarify that topic to myself.</p>
<p>I have my personal knowledge base. Almost all notes there are packs of links and quotes. To make them useful I rewrite them from time to time. Some of them become blog posts because they are important to me. I spend a lot of time writing and rewriting them before they become blog posts.</p>
<p>I have a second minor goal for this blog. I use it to boost my career. Having a public presence is a must nowadays. Some developers have great GitHub profiles, others have gazillion followers on twitter. I prefer having a blog. It is the most comfortable format for me.</p>
<h2>Share buttons</h2>
<p>First candidates to delete are share buttons. I used to have them under a post. And people used them. I had a couple of shares for almost each blog post I’ve posted here. But not enough to justify having those buttons.</p>
<p>I used to use buttons from one quite popular service which provides a share-anywhere block of buttons. I tried to understand whether they are GDPR friendly. I found no clear answer. So I cut them out from the post template.</p>
<h2>Analytics</h2>
<p>The most difficult decision was to delete the analytics. I’m used to having it on all my websites since… the Big Bang. It is kind of a habit: if I start a new website, I add analytics on it.</p>
<p>However, I didn’t use it. I had opened the analytics report for this blog right before I cut it out. Before that I opened it half a year ago. Not much usage of the data, isn’t it? If I don’t use it, why on Earth do I collect it?</p>
<h2>Comments</h2>
<p>My oldest post on this blog was posted in 2016. Since then people have left few comments (about 10 in total). I’ve received many more questions on Twitter and on LinkedIn.</p>
<p>I don’t have much trouble with spam. Disqus has really good spam-protection. But again, I don’t really need Disqus. Like <a href="https://strobist.blogspot.com/2013/12/why-i-moved-comments-to-twitter.html">Strobist</a> I’ll get more value starting discussions on Twitter.</p>
<p>Let’s discuss this on <a href="https://twitter.com/avkorablev/status/1529119534019379200">Twitter</a>.</p>
<p><strong>P.S.</strong> There are not many articles from anyone who decided to run their blogs without analytics or comments. I found this two links very useful:
- <a href="https://ajenkins.me/post/analytics">Why I don’t have analytics on my website</a>
- <a href="https://strobist.blogspot.com/2013/12/why-i-moved-comments-to-twitter.html">Strobist: Why I Moved Comments to Twitter</a></p>Using typing.Generic in Python2022-02-18T10:00:00+05:002022-02-18T10:00:00+05:00akorablevtag:www.alexkorablev.com,2022-02-18:/generics-in-python.html<p>I am working on a project with a relatively large code-base. And it has a history.
Our team wrote some parts of it way before type-hints. And we steel add hints to the
legacy code or improve them. Is it worth the fuss? Sure! Our users are developers.
They open our code in PyCharm every day. And they hope that it will help them to solve
their task as quickly as possible.</p>
<p>I am working on a project with a relatively large code-base. And it has a history.
Our team wrote some parts of it way before type-hints. And we steel add hints to the
legacy code or improve them. Is it worth the fuss? Sure! Our users are developers.
They open our code in PyCharm every day. And they hope that it will help them to solve
their task as quickly as possible.</p>
<p>I can say that there is a correlation between how precise code completion is and how
fast developers work. However, our goal is not only to introduce more precise static
analysis and code completion in PyCharm but not to break their production code.</p>
<p>"not to break their production code" means that we can't change class names and their API
if it is not necessary. And type-hint is not that case.</p>
<p>One of my colleagues added a hint to the code that looks like this:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_0.py</span>
<span class="k">class</span> <span class="nc">A</span><span class="p">:</span>
<span class="n">a</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">b</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="k">class</span> <span class="nc">DoSomethingWithA</span><span class="p">:</span>
<span class="n">_class</span> <span class="o">=</span> <span class="n">A</span>
<span class="k">def</span> <span class="nf">do</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">A</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_class</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">DoSomethingWithB</span><span class="p">(</span><span class="n">DoSomethingWithA</span><span class="p">):</span>
<span class="n">_class</span> <span class="o">=</span> <span class="n">B</span>
</code></pre></div>
<p>PyCharm is OK with that code. Its analyzer shows a green checkmark. Mypy doesn't show any errors either:</p>
<div class="highlight"><pre><span></span><code>$ mypy type_cast_0.py
Success: no issues found <span class="k">in</span> <span class="m">1</span> <span class="nb">source</span> file
</code></pre></div>
<p>Let’s add a code that uses <code>DoSomethingWithB</code>.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_1.py</span>
<span class="kn">from</span> <span class="nn">type_cast_0</span> <span class="kn">import</span> <span class="n">DoSomethingWithB</span>
<span class="nb">print</span><span class="p">(</span><span class="n">DoSomethingWithB</span><span class="p">()</span><span class="o">.</span><span class="n">do</span><span class="p">()</span><span class="o">.</span><span class="n">b</span><span class="p">)</span>
</code></pre></div>
<p>PyCharm shows a warning immediately: <code>Unresolved attribute reference 'b' for class 'A'</code>.
Mypy also marks it as an error.</p>
<div class="highlight"><pre><span></span><code>$ mypy type_cast_1.py
type_cast_1.py:4: error: <span class="s2">"A"</span> has no attribute <span class="s2">"b"</span>
Found <span class="m">1</span> error <span class="k">in</span> <span class="m">1</span> file <span class="o">(</span>checked <span class="m">1</span> <span class="nb">source</span> file<span class="o">)</span>
</code></pre></div>
<p>Let’s fix it. Here is my first approach. It is a naive use of generics in Python.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_2.py</span>
<span class="c1">#...</span>
<span class="n">TV</span> <span class="o">=</span> <span class="n">tp</span><span class="o">.</span><span class="n">TypeVar</span><span class="p">(</span><span class="s1">'TV'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">DoSomethingWithA</span><span class="p">(</span><span class="n">tp</span><span class="o">.</span><span class="n">Generic</span><span class="p">[</span><span class="n">TV</span><span class="p">]):</span>
<span class="n">_class</span><span class="p">:</span> <span class="n">tp</span><span class="o">.</span><span class="n">Type</span><span class="p">[</span><span class="n">TV</span><span class="p">]</span> <span class="o">=</span> <span class="n">A</span>
<span class="k">def</span> <span class="nf">do</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">TV</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_class</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">DoSomethingWithB</span><span class="p">(</span><span class="n">DoSomethingWithA</span><span class="p">):</span>
<span class="n">_class</span> <span class="o">=</span> <span class="n">B</span>
</code></pre></div>
<p>PyCharm shows no errors or warnings. But mypy does.</p>
<div class="highlight"><pre><span></span><code>$ mypy type_cast_3.py
type_cast_2.py:17: error: Incompatible types <span class="k">in</span> assignment <span class="o">(</span>expression has <span class="nb">type</span> <span class="s2">"Type[A]"</span>, variable has <span class="nb">type</span> <span class="s2">"Type[TV]"</span><span class="o">)</span>
Found <span class="m">1</span> error <span class="k">in</span> <span class="m">1</span> file <span class="o">(</span>checked <span class="m">1</span> <span class="nb">source</span> file<span class="o">)</span>
</code></pre></div>
<p>Interesting… Trying to change <code>TV = tp.TypeVar('TV')</code> to <code>TV = tp.TypeVar('TV’, bound=A)</code>.
Same error. More interesting…</p>
<p>Official python documentation isn’t helpful in this case. There are a couple of
examples of <a href="https://docs.python.org/3/library/typing.html#typing.Generic">using Generics</a>, but
nothing that gives an idea of how to fix the issue. Hopefully,
there is a <a href="https://mypy.readthedocs.io/en/stable/generics.html">brilliant section</a> about
Generics on the mypy website.</p>
<p>For my example, the resulting code would be like this one.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_6.py</span>
<span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">DoSomethingWith</span><span class="p">(</span><span class="n">tp</span><span class="o">.</span><span class="n">Generic</span><span class="p">[</span><span class="n">TV</span><span class="p">]):</span>
<span class="n">_class</span><span class="p">:</span> <span class="n">tp</span><span class="o">.</span><span class="n">Type</span><span class="p">[</span><span class="n">TV</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">do</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">TV</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_class</span><span class="p">()</span>
</code></pre></div>
<p>Here is a usage example.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_7.py</span>
<span class="kn">from</span> <span class="nn">type_cast_6</span> <span class="kn">import</span> <span class="n">DoSomethingWith</span><span class="p">,</span> <span class="n">B</span>
<span class="nb">print</span><span class="p">(</span><span class="n">DoSomethingWith</span><span class="p">[</span><span class="n">B</span><span class="p">]()</span><span class="o">.</span><span class="n">do</span><span class="p">()</span><span class="o">.</span><span class="n">b</span><span class="p">)</span>
</code></pre></div>
<p>Mypy shows no problems. Neither does PyCharm.</p>
<div class="highlight"><pre><span></span><code>$ mypy type_cast_6.py
Success: no issues found <span class="k">in</span> <span class="m">1</span> <span class="nb">source</span> file
$ mypy type_cast_7.py
Success: no issues found <span class="k">in</span> <span class="m">1</span> <span class="nb">source</span> file
</code></pre></div>
<p>But actual usage of the code throws an exception.</p>
<div class="highlight"><pre><span></span><code>$ python type_cast_7.py
...
AttributeError: <span class="s1">'DoSomethingWith'</span> object has no attribute <span class="s1">'_class'</span>
</code></pre></div>
<p>Unfortunately, there is no way to use TypeVar as you can use generics in Java, for example.
So I can’t assign <code>TV</code> to <code>_class</code> and expect from Python that it replace type variable to actual
class during the execution. In other words, if I use <code>_class: tp.Type[TV] = TV</code> in <code>type_cast_6.py</code>,
I’ll get an <code>TypeError: 'TypeVar' object is not callable</code>.</p>
<p>To avoid that I add subclasses for <code>DoSomethingWithz</code>.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_8.py</span>
<span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">DoSomethingWithA</span><span class="p">(</span><span class="n">DoSomethingWith</span><span class="p">):</span>
<span class="n">_class</span> <span class="o">=</span> <span class="n">A</span>
<span class="k">class</span> <span class="nc">DoSomethingWithB</span><span class="p">(</span><span class="n">DoSomethingWith</span><span class="p">):</span>
<span class="n">_class</span> <span class="o">=</span> <span class="n">B</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1"># file name: type_cast_9.py</span>
<span class="kn">from</span> <span class="nn">type_cast_8</span> <span class="kn">import</span> <span class="n">DoSomethingWithB</span>
<span class="nb">print</span><span class="p">(</span><span class="n">DoSomethingWithB</span><span class="p">()</span><span class="o">.</span><span class="n">do</span><span class="p">()</span><span class="o">.</span><span class="n">b</span><span class="p">)</span>
</code></pre></div>
<p>Not particularly elegant, but it works.</p>
<p>This post has a lot of code examples. I truncated them a bit. You can find full versions of the
examples on <a href="https://github.com/avkorablev/code_4_blog/tree/main/type_cast">my GitHub account</a>.</p>How to Improve Your Whiteboard Coding Interview Skill2021-10-18T10:00:00+05:002021-10-18T10:00:00+05:00akorablevtag:www.alexkorablev.com,2021-10-18:/whiteboard-interview-skill.html<p>In the last couple of years, I have conducted more than 30 job interviews: on-side whiteboard interviews and, because of Covid, Skype interviews. I saw candidates with different skills. I saw two superstars of whiteboard coding. I saw many guys who thought that they can code, but weren’t able to write a single line or tell <code>for</code> from <code>while</code>. But these are corner cases. What about a regular guy with enough skill and knowledge for the job but not so experienced in whiteboard programming? Like me. I have given much fewer interviews than I have taken. So I am a junior in whiteboard coding.</p>
<p>As an interviewer, however, I can give you one piece of advice that will double your chances to get a job. Here is my tip: <strong>Write down every state of each variable in your code during a whiteboard interview.</strong> That is it. You can stop watching here. I have already given you the main idea of this video. For the rest of it, I am going to explain why it is so important and how to develop that skill.</p>
<p>In the last couple of years, I have conducted more than 30 job interviews: on-side whiteboard interviews and, because of Covid, Skype interviews. I saw candidates with different skills. I saw two superstars of whiteboard coding. I saw many guys who thought that they can code, but weren’t able to write a single line or tell <code>for</code> from <code>while</code>. But these are corner cases. What about a regular guy with enough skill and knowledge for the job but not so experienced in whiteboard programming? Like me. I have given much fewer interviews than I have taken. So I am a junior in whiteboard coding.</p>
<p>As an interviewer, however, I can give you one piece of advice that will double your chances to get a job. Here is my tip: <strong>Write down every state of each variable in your code during a whiteboard interview.</strong> That is it. You can stop watching here. I have already given you the main idea of this video. For the rest of it, I am going to explain why it is so important and how to develop that skill.</p>
<h2>Why is it so important? A sad story about why we have to code on a whiteboard...</h2>
<p>If you ask me, a coding interview is a waste of time. As a candidate, you have to show skills that don’t require for a position you want to get. As an interviewer, you ask people to solve problems that have nothing to do with the real world and make them struggle to balance a Red-Black tree... But there is a company. From its point of view, a whiteboard interview is not a bad thing. </p>
<p>Companies want to hire someone to fill the position. They want to hire someone as soon as possible. But costs of hiring the wrong person are much higher than the costs of missing the right one. To fit corporate processes, the method should be quite cheap and highly scalable. And it should filter out as many wrong persons as possible. As a result, we have to have that stupidly long series of coding interviews (Stupidly long is more than one).</p>
<p>Whiteboard coding is one of many options. There are challenges for homework, a day of work in a team on real-world tasks to name a few. These methods are better ways to choose a developer. But coding interviews are much cheaper and they work quite well. They filter out candidates who can’t code at all.</p>
<p>But because its efficiency is far from ideal, companies tend to chain coding interviews. The bigger the company the more chances there are several whiteboard coding interviews. In today's industry, the only chance to avoid this kind of interview is being a superstar developer who is trying to find a job in a small company.</p>
<h2>Failure Rate</h2>
<p>From my experience, only one of 8 candidates successfully performed during the interview. At least two candidates out of the 8 who failed had a chance to do their interview well. Why is the rate so low? I have analyzed many interviews to find a pattern that leads a candidate to success. But first of all, let's look at why candidates fail.</p>
<p>The main reason is that candidate has no coding skills. Unfortunately, it happens often. The stronger the HR brand of the company the more often it happens. Usually, candidates from that category have no chance to get an offer. Usually, they can't write a line of code. The second reason for failure is a candidate's inability to debug their code on a whiteboard. All candidates who performed well, especially superstars, all debug their code.</p>
<p>I’ll show you a couple of scenarios on how I as a candidate can debug my code. From the worst to the best.</p>
<p>The worst scenario is quite short. </p>
<p><strong>Me:</strong> “It looks OK”<br>
<strong>Interviewer:</strong> “No. There are bugs”</p>
<p>Some random changes in the code… </p>
<p><strong>Me:</strong> “Is it OK now?”<br>
<strong>Interviewer:</strong> “No…”</p>
<p>Usually, there is one more iteration of this… discussion. As a result, the reviewer texts an HR “No hire”. This scenario is quite common. Often it happens when a candidate is in a hurry or a task is too complicated. The best outcome from this interview is not an offer, but a chance to have another attempt in 6 months or a year. So maybe there is a better strategy than just saying nothing?</p>
<p>The next scenario is better although far from the ideal. </p>
<p>The candidate debugs the code but writes nothing down. For an interviewer, it is extremely difficult to follow a candidate’s thoughts. And it is error-prone. From my experience mistakes that candidate makes in the debug stage ruin the whole interview more often than a code full of bugs. Sometimes it looks like a joke. </p>
<p>Once I took an interview with one guy. I gave him a problem. He wrote a decent code. Few bugs, nothing special. He started debugging. He wrote nothing down. He made a mistake once. If I remember it right, he forgot the previous value of an index. First strike. He made a mistake again. Again something with indexes. Second strike. Finally, he solved the problem. I gave him the next problem to solve. Again he wrote a decent code with little help. He knew how to code. Debugging stage… </p>
<p>The candidate started going through his code using a test case from the description of the problem. It was a long test case. He had to go 9 times through the cycle in the code. It required a lot of time. Of course, he stopped after 3 iterations saying “It works”. No, it didn’t. There was a bug in the first line of the cycle and the very next step would show it. But… No hire!</p>
<p>Anyway, a couple of times candidates passed their interviews with that style of debugging. No one passed the next stage with a bit more advanced tasks.</p>
<p>In the third scenario, you write down every state of each variable in your code during debugging. Again it is the story from my experience. A candidate less experienced in coding than the previous one wrote a code full of bugs. However, during debugging he wrote everything down. Every value of each variable was on the board. As a result, he found all bugs in his code. And did it quite fast.</p>
<h2>Example</h2>
<p>What did he do? To illustrate that I will solve a simple programming problem. Let’s assume that I am a candidate and I was asked to solve the <a href="https://www.hackerrank.com/challenges/counting-valleys/problem">Counting Valleys</a> from HackerRank.</p>
<blockquote>
<p>An avid hiker keeps meticulous records of their hikes. During the last hike that took exactly steps, for every step, it was noted if it was an uphill, or a downhill, step. Hikes always start and end at sea level, and each step up or down represents a unit change in altitude. We define the following terms:</p>
<p>A mountain is a sequence of consecutive steps above sea level, starting with a step up from sea level and ending with a step down to sea level.</p>
<p>A valley is a sequence of consecutive steps below sea level, starting with a step down from sea level and ending with a step up to sea level.</p>
<p>Given the sequence of up and down steps during a hike, find and print the number of valleys walked through.</p>
</blockquote>
<p>There are two sample inputs.</p>
<div class="highlight"><pre><span></span><code><span class="k">assert</span> <span class="n">counting_valleys</span><span class="p">(</span><span class="s1">'UDDDUDUU'</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">assert</span> <span class="n">counting_valleys</span><span class="p">(</span><span class="s1">'DDUUDDUDUUUD'</span><span class="p">)</span> <span class="o">==</span> <span class="mi">2</span>
</code></pre></div>
<p>After a while, I wrote this code. Probably, there are some bugs in this code. During an interview, I will assume that I always write with bugs.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">counting_valleys</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="n">level</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">valleys</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">s</span><span class="p">:</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">'D'</span><span class="p">:</span> <span class="c1"># Down-if</span>
<span class="n">level</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">level</span> <span class="o">==</span> <span class="o">-</span><span class="mi">1</span><span class="p">:</span> <span class="c1"># Level-if</span>
<span class="n">valleys</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">'U'</span><span class="p">:</span> <span class="c1"># Up-if</span>
<span class="n">level</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="n">valleys</span>
</code></pre></div>
<p>First of all, I will write down all variable names from the code in the list near it.</p>
<p><img alt="" src="https://www.alexkorablev.com/images/whiteboard-interview-skill-01.png"></p>
<p>It is wise to prepare a couple of test cases in advance that is short and cover all corner cases. But it is a topic for a different discussion. I have prepared one. It is a bad test case. I would prefer a shorter one. I made it longer than necessary to illustrate the process. </p>
<h3>Let’s go</h3>
<div class="highlight"><pre><span></span><code><span class="k">assert</span> <span class="n">counting_valleys</span><span class="p">(</span><span class="s1">'UDDDU'</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
</code></pre></div>
<p>So <code>s</code> is equal to <code>UDDDU</code>
<code>level</code> and <code>valleys</code> are equal to <code>0</code>
The cycle will be evaluated because <code>s</code> is not empty
<code>c</code> is equal to <code>U</code>. I will underline the first <code>U</code> in <code>s</code>.
The condition in <code>Down-if</code> is <code>False</code>. Move to the <code>Up-if</code>. Condition is <code>True</code>. <code>level</code> is increased by one. <code>level</code> equals <code>1</code>.</p>
<p><img alt="" src="https://www.alexkorablev.com/images/whiteboard-interview-skill-02.png"></p>
<p>Next iteration.</p>
<p><code>c</code> is equal <code>D</code>. Underline corresponding <code>D</code> in <code>s</code>
First <code>if</code>. The condition is <code>True</code>.
<code>level</code> is now equal <code>0</code>
So condition <code>level == -1</code> is <code>False</code></p>
<p><img alt="" src="https://www.alexkorablev.com/images/whiteboard-interview-skill-03.png"></p>
<p>Next iteration.</p>
<p>Again <code>c</code> is equal <code>D</code>. Underline corresponding <code>D</code> in <code>s</code>
First <code>if</code>. The condition is <code>True</code>.
<code>level</code> is now equal <code>-1</code>
So condition <code>level == -1</code> is <code>True</code>
<code>valleys</code> is now equals <code>1</code></p>
<p><img alt="" src="https://www.alexkorablev.com/images/whiteboard-interview-skill-04.png"></p>
<p>Hm… It looks like we have found a bug. So I will move the code <code>Level-if</code> block inside <code>Up-if</code> below increasing the level. And I will change the condition to <code>level == 0</code></p>
<p>So this is my final code:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">counting_valleys</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="n">level</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">valleys</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">s</span><span class="p">:</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">'D'</span><span class="p">:</span> <span class="c1"># Down-if</span>
<span class="n">level</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">'U'</span><span class="p">:</span> <span class="c1"># Up-if</span>
<span class="n">level</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">level</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="c1"># Level-if</span>
<span class="n">valleys</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="n">valleys</span>
</code></pre></div>
<p>I've run the test case over it. My final state of the list of variables is:</p>
<p><img alt="" src="https://www.alexkorablev.com/images/whiteboard-interview-skill-05.png"></p>
<p>I have checked this solution on HackRank. It is OK. All tests are green. As you can see it is much easier to follow me. We found a bug. Of course, it was an extremely simple problem. But with more complex problems this approach works even better. During the interview, it will help both an interviewer and you. Interviewers usually have the ideal - or what they think is the ideal - solution for a problem they give you. If you come up with a different solution, you have to prove that it works and it is good enough. </p>
<p>The main drawback of this approach is that it is very difficult to practice, at least for me. Usually, I use sites like Hackerank to find problems to solve. There is an online editor. It is so tempting to start code there. I prefer to start with pen and paper. I can decompose the problem easily. I can draw. I can write a couple of drafts. The next step could be finding some test cases… And debugging on that piece of paper. </p>
<p>But because I am a quite lazy person I switch back to the online editor too early. From time to time I even solve a problem in front of a computer from start to finish. It helps me to improve problem-solving skills, but it doesn’t help to improve interview skills.</p>
<p>If you are like me tend to do the same, I can suggest a trick that works for me more or less OK while I work on a problem from Hackerrank or a similar website. </p>
<ul>
<li>I write down significant parts of the problem using my own words. </li>
<li>Write down all given test cases if there are any as a reference. </li>
<li>I go to a quiet place without my laptop. This is the crucial part. Left the laptop away.</li>
<li>Sit down there and solve the problem… with inventing my test cases, debugging, and if it is possible I do it out loud. </li>
<li>And only if I have a solution, I return to my computer and type it in. </li>
</ul>
<p>There is a mailing list called Daily Coding Problem. If you are OK without checking your solution with a set of tests online, using that list could be a good alternative. Or you can organize many interviews for yourself. It is rather easy to do. </p>
<h2>To summarise the idea</h2>
<p>Assume that your code has bugs. Assume that an interviewer has in mind a different solution for a problem. Assume that neither interviewer nor you can keep more than one state of one variable in memory. Write everything down. Every state, every variable of each variable you have in your code. And practice, practice, practice... </p>
<p>P.S. Does anybody can recommend a decent book for coding interview preparation? I know only Cracking the Coding Interview. </p>What are good Python interview questions?2021-05-14T10:00:00+05:002021-05-14T10:00:00+05:00akorablevtag:www.alexkorablev.com,2021-05-14:/python-interview-questions.html<p>There are no good Python interview questions. Why? Because you want to hire a good developer, not a Python nerd. I used to think that there are some good Python-related problems to solve during the interview. But I doubt there is any.</p>
<p>I've interviewed more than 30 developers. I've gradually drifted from only-python-nerd-knows-the-answer questions to general programming problems to solve. There are several reasons for that.</p>
<p>There are no good Python interview questions. Why? Because you want to hire a good developer, not a Python nerd. I used to think that there are some good Python-related problems to solve during the interview. But I doubt there is any.</p>
<p>I've interviewed more than 30 developers. I've gradually drifted from only-python-nerd-knows-the-answer questions to general programming problems to solve. There are several reasons for that.</p>
<p>An interview is not an ideal thing. You have an hour to decide if a candidate suits a position. In most cases, you as an interviewee look at a particular skill - task solving skill. This skill should be accompanied by the ability to code and test own code. These are the second and the third skills to check.</p>
<h2>Maybe, ask how GIL works?</h2>
<p>So what is the popular Python-related question? How does GIL work? Maybe. At least I saw that abbreviation a lot in lists of interview questions.</p>
<p>Let's imagine a typical interview. The candidate, John, is a brilliant guy with many years of experience. John had spent about 20 minutes describing GIL in detail. Wonderful. But what do I learned from the answer about his ability to code? Not much if anything.</p>
<p>I can try to find a coding problem where GIL is a bottleneck and ask John to solve it. But I can't. Real-world problems connected with GIL usually require at least a week to solve. And mostly, they are architecture-related issues. </p>
<p>That's why questions about GIL or CPython implementation details are wastes of time. Answering them, a candidate doesn't show any required skills. So why interviewers keep asking them?</p>
<h2>What about the standard library?</h2>
<p>Python has a huge standard library. There is a trap if someone thinks that there is a commonly used part. Even build-in functions like "sort" are not used often. As an example, last year I hardly used it.</p>
<p>Anyway, Python has probably the best documentation. So asking this kind of question is equivalent to asking someone to recall the part of the documentation.</p>
<h2>Ok. What if I ask a question about syntax sugar?</h2>
<p>Decorators are told to be a good thing to ask during an interview. I don't think so. Any engineer with a non-Python background will fail on this kind of question. But it takes 10 minutes to explain the basics and extra 15-20 minutes to explain decorators in depth. </p>
<p>I used to ask about decorators. If interviewees have experience in programming, usually they can solve decorator-based problems after a 5-minute-long decorator crash course.</p>
<h2>As a conclusion</h2>
<p>Python is not the hardest language to learn. If you can't invite an interviewee to work with you on a real task, a real-world feature in your project for a couple of days, and you have to use traditional technical interviews, use general programming problems.</p>
<p>It is much better to filter out a guy who can't tell "for" from "while" than a guy who can't explain how GIL works in detail.</p>Audio Programming Frameworks2020-04-10T10:00:00+05:002020-04-10T10:00:00+05:00akorablevtag:www.alexkorablev.com,2020-04-10:/audio-frameworks.html<p>Music production has been my hobby for a year now. It's the main reason why I wasn't so active in blogging last year. Now I am used to it. I've developed some new habits and balanced my life. And I think it's high time to bring some programming into it. </p>
<p>I'm going to program a couple of audio plugins or standalone applications. I have several ideas to implement. And I'm going to write a series of articles about audio programming.</p>
<p>Music production has been my hobby for a year now. It's the main reason why I wasn't so active in blogging last year. Now I am used to it. I've developed some new habits and balanced my life. And I think it's high time to bring some programming into it. </p>
<p>I'm going to program a couple of audio plugins or standalone applications. I have several ideas to implement. And I'm going to write a series of articles about audio programming.</p>
<p>This post is an overview of the current state of audio programming frameworks. What options developers have to start with and my choice. It is quite short because I haven't found many popular frameworks.</p>
<h1>DIY</h1>
<p>You can start a new audio processing project without any framework. I doubt it is a wise decision. There are too many boring things to do before you can start processing audio. To name a few:</p>
<ul>
<li>you have to have GUI with knobs and sliders</li>
<li>you have to implement a lot of OS or plugin-specific functions (file handling, getting audio buffers and so on)</li>
<li>you have to implement a lot of utility functions.</li>
</ul>
<p>This approach is OK only if you want to make everything by yourself. It is a quite risky strategy. There is a good chance that you can lose focus and end up without enough energy to finish a project.</p>
<h1>Game Engine</h1>
<p>On some degree, audio plugins are similar to games. GUI isn't required to be native for any platform. I can say it hardly ever has native widgets or controls. Sometimes audio plugins have weird interfaces that work only because they inspire their users. Of course, some amazingly sounding plugins are completely unusable because of UX errors.</p>
<p>Depending on the idea of the project it can be possible to even use a game development framework like Unity or Unreal engine. They both have strong audio subsystems. They both have a good library to build a decent cross-platform GUI. And you can easily ship your project to plenty of platforms.</p>
<p>There are some experiments in this area. There is a software synthesizer for Unity: Helm. It is a fully functional digital software instrument. It worth checking it out. I'm not sure about licensing issues of using it in commercial software, but you definitely can experiment with it. I think it is a good starting point for music learning software with elements of gamification for example.</p>
<h1>Specialized Framework</h1>
<p>If your goal is to develop a plugin, you should take into account that nowadays any audio plugin should be cross-platform. The market demands it. Many producers use Windows laptops. Others use Mac. Both markets are big enough. However, the plugin only for one platform looks a bit amateurish... </p>
<p>As a producer, I don't want to use Mac only third-party plugins. I can't use Windows only plugins. I'm quite OK with Mac only DAW (Logic pro from Apple) but only because I have a backup plan and it costs only $200.</p>
<p>And if a plugin exists only for one platform, it looks like it is a hobby project. There is no reason for that, but many producers just avoid using such plugins unconsciously.</p>
<p>There is only one exception. In the case of iOS plugin, it's OK to be for that mobile platform only. Because iOS is the only mobile platform with plugins.</p>
<p>Here is my shortlist of specialized frameworks for audio programming. Of course, bare C++ is an option, but as I have already mentioned, it requires a lot of boring work.</p>
<p>At the moment there are only two frameworks in the list. And one matches all my requirements. Let's start with it.</p>
<h2>JUCE</h2>
<p>The first option is a popular C++ framework for audio applications called <a href="https://juce.com/">JUCE</a>. Its selling point is cross-platform. It's possible to use it for desktop and mobile development.</p>
<p>It has a GUI library (based on OpenGL) and the possibility to make both standalone applications and plugins. There is a lot of audio-related stuff like DSP (digital signal processor) classes.</p>
<p>As examples of applications that are build using this framework are BeatWalk and Spacecraft. JUCE is free for small projects (in terms of money).</p>
<h2>Audio Kit</h2>
<p>Another framework for audio programming is <a href="https://audiokit.io/">Audio Kit</a>. It's for the Apple ecosystem only. Despite this limitation, it looks interesting.</p>
<p>You can use Swift. You can use native controls. And the DSP library is more interesting than JUCE's.</p>
<p>It can be OK for some projects. For example, if you want to make a plugin for mobile. The only iOS has such thing as audio plugins. So you don't have to care about cross-platform.</p>
<p>Not much. I hope there are more frameworks. Anyway, two actively developed frameworks with the active community are better than 10 less popular options.</p>
<p>For my experiments, I go for JUCE. Mainly because of C++. Many examples and sample implementations of algorithms are in C++. It's much easier to copy-paste things and then find a creative way to make something sounding great combining them.</p>Type Hints for Dynamic Class Creation2020-03-15T10:00:00+05:002020-03-15T10:00:00+05:00akorablevtag:www.alexkorablev.com,2020-03-15:/cast-function.html<p>Type hints in Python are cool. Projects of any size can benefit from them. Actually for projects bigger than 50-60 modules or with more than 5 developers involved type hints become crucial to keep its code quality at a reasonable level. And it looks like it is my favorite topic to write about.</p>
<p>But sometimes, its realization in Python is counterintuitive or even confusing. You have to write a code that has no sense but typing, for example, in case of dynamic type creation and necessity of using <code>cast</code> function. Well, technically all type hints in Python have no sense but typing. However, in the vast majority of cases, type annotation adds clarity to the code, not confusion.</p>
<p>Type hints in Python are cool. Projects of any size can benefit from them. Actually for projects bigger than 50-60 modules or with more than 5 developers involved type hints become crucial to keep its code quality at a reasonable level. And it looks like it is my favorite topic to write about.</p>
<p>But sometimes, its realization in Python is counterintuitive or even confusing. You have to write a code that has no sense but typing, for example, in case of dynamic type creation and necessity of using <code>cast</code> function. Well, technically all type hints in Python have no sense but typing. However, in the vast majority of cases, type annotation adds clarity to the code, not confusion.</p>
<p>Long story short. Recently one of my colleges has implemented a new feature in our project. As part of its design there is quite a lot of function that looks like that:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Type</span>
<span class="k">class</span> <span class="nc">BaseClass</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">create_subclass</span><span class="p">(</span><span class="o">**</span><span class="n">args</span><span class="p">)</span> <span class="o">-></span> <span class="n">Type</span><span class="p">[</span><span class="n">BaseClass</span><span class="p">]:</span>
<span class="k">return</span> <span class="nb">type</span><span class="p">(</span>
<span class="s1">'SubClass'</span><span class="p">,</span>
<span class="p">(</span><span class="n">BaseClass</span><span class="p">,),</span>
<span class="n">args</span>
<span class="p">)</span>
</code></pre></div>
<p>The only problem with this code is that PyCharm marks it as not correct type-wise: <code>Expected type 'Type[BaseClass]', got 'type' instead</code>. Autocompletion works. But this highlighted block stops me every time I see it. mypy doesn't find any problems with that code showing <code>Success: no issues found in 1 source file</code>. That looks right.</p>
<blockquote>
<p>It looks right, but mypy does nothing with that code. I made an additional experiment with it. Even if I put <code>WrongBase</code> instead of <code>BaseClass</code> in type function, it doesn't find any problem.</p>
</blockquote>
<p>I thought there was a typed alternative for <code>type</code> function. Like one we have for named tuples. There is no such thing in the typing library. Probably the main reason is that <code>type</code> in this context isn't used often.</p>
<p>However, the construction itself looks like its type can be calculated automatically. At least it can be true for some cases. Why? Because of the <code>type</code> function definition. We have a tuple with base types. So, as a result, we get a subtype of them. </p>
<p>This statement makes sense only if you have only one base class and do not add extra methods to the subclass. Before evaluating type function you don't have a subclass. It is impossible to annotate the result. In the simplest case (one base, no extra things changing class interface) it is possible to say that the result has the type equals to the base class.</p>
<p>Sometimes even if you have some changes, it is Ok to say that the type of this construction is equal to the base class. Often type is used to make some new classes with a predefined interface.</p>
<p>However in the case of multiple base classes it hard to say what the type hint should be used. I don't know how to write a type hint for construction with two or more superclasses. <code>Union</code>?</p>
<p>And <code>type</code> creates a new class. You can bounce methods that change the public interface. This construction can get a list of basses that have been built dynamically. And it makes automatic type hints next to impossible here. </p>
<p>So we have to find a workaround. The only way I've found is to use the <code>cast</code> function. It solves the problem because we have only one base class and can use it as a <code>cast</code> parameter. But it looks a bit strange in the code.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Type</span><span class="p">,</span> <span class="n">cast</span>
<span class="k">class</span> <span class="nc">BaseClass</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">create_subclass</span><span class="p">(</span><span class="o">**</span><span class="n">args</span><span class="p">)</span> <span class="o">-></span> <span class="n">Type</span><span class="p">[</span><span class="n">BaseClass</span><span class="p">]:</span>
<span class="k">return</span> <span class="n">cast</span><span class="p">(</span>
<span class="n">Type</span><span class="p">[</span><span class="n">BaseClass</span><span class="p">],</span>
<span class="nb">type</span><span class="p">(</span><span class="s1">'SubClass'</span><span class="p">,</span> <span class="p">(</span><span class="n">BaseClass</span><span class="p">,),</span> <span class="n">args</span><span class="p">)</span>
<span class="p">)</span>
</code></pre></div>
<p>The <code>cast</code> function is similar to <code>dynamic_cast</code> in C++: let's pretend that this is an instance of A and prey that it is always true. But in Python, we don't worry about it. Firstly, it does nothing, just mark its argument with a given type. Secondly, if something has the required attribute it matches. It can cause errors and hard-to-find bugs, but it's a different story. And it is the way how Python and other interpreted languages work.</p>
<p>There are a lot of typing hint constructions that look strange. Mainly because of the optionality of typing in Python. However, type hinting solidifies big Python projects. I can't stress enough how important is to use them in your everyday work. Yes, it will fight common sense and dynamic Python nature from time to time, but it worth using them.</p>Is It Worth Become a Developer?2020-01-20T10:00:00+05:002020-01-20T10:00:00+05:00akorablevtag:www.alexkorablev.com,2020-01-20:/new-carrier.html<p>There is a whole week of bank holidays at the very beginning of January in Russia. Sometimes it causes strange thoughts and ideas in people's minds. For example two of my friends decided to become programmers. </p>
<p>There is a whole week of bank holidays at the very beginning of January in Russia. Sometimes it causes strange thoughts and ideas in people's minds. For example two of my friends decided to become programmers. </p>
<p>There is nothing wrong with that. It is quite easy to find a good job with a decent salary even if you have little experience. And it's super easy to do that if you have a couple of years of experience. </p>
<p>However, their way of pursuing this goal is completely wrong. They both have decided that they have to spend a lot of money on courses missing one thing about our industry. You can try it before "buy" it.</p>
<p>Our industry is not medicine. You don't have to learn many years before you can practice. In programming, you can start immediately. Find and idea and try to implement it in code. That's it. It can make your education easier if you have someone to validate your idea or to help with coding. But it is not necessary. </p>
<p>I want to say that good enough coding skill is just a matter of time and energy. And actually, formal education or online courses don't help to get them. The only way to learn how to code is coding itself. Your first project doesn't need to be useful and big. It could be a small useless thing. For me, it was implementing Conway's Game of Life in Pascal.</p>
<p>The main goal for your first project should be understanding software development is for you or not. I guess it is not wise to change one carrier you not happy with another that is not for you.</p>
<p>All software developers I know have chosen their carriers not because it was a wise decision but because it was the only possible way for them. Almost all of them started programming when they were teenagers. At least a couple of years before university. Several guys changed jumped into programming from different fields. However, they wrote code for a while before that.</p>
<p>So how fast you can get your first programmer job? Well... Half a year before getting an internship is enough. But if you have a family to support it would take at least two years to get a salary big enough to leave your current job. In this case, you have to build a developer portfolio full of interesting projects.</p>
<p>Probably it can vary from country to country. But as far as I know, an intern isn't the best-paid position. Junior developer earns more. But to get this position you have to have some experience.</p>
<p>I can imagine that someone can spend half a year learning programming without passion. But I doubt that anyone can do it for two years. And in any what's the point to do that? It's better to find something else.</p>
<p>And the last point. If you are absolutely sure that you want to ditch your carrier to software development, use it as your strength. You will compete with young guys who have years of experience in the field even if they have just graduated. </p>
<p>But your current carrier can help you to differentiate yourself. It can be a unique business experience, deep knowledge of a particular industry or soft skills.</p>What is Python not good for?2019-10-15T10:00:00+05:002019-10-15T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-10-15:/what-is-python-not-good-for.html<p>There is a question on Quora - "What is Python not good for?" I want to say that Python is good for everything, but it is not true. Honestly, Python is a second-best language for many things. There are not many spots there it is the best language to use.</p>
<p>There is a question on Quora - "What is Python not good for?" I want to say that Python is good for everything, but it is not true. Honestly, Python is a second-best language for many things. There are not many spots there it is the best language to use.</p>
<p>For example, I think Go is the best language for web services. It's easy to learn, it's much easier to distribute. However, Python is the second-best option. Event for microcontrollers I would think of using Python not C/C++ because of MicroPython. Well, MicroPython is technically not Python, but it's close enough to call it Python.</p>
<p>In data science and machine-learning Python is number one.</p>
<p>There are several reasons for that. First of all, Python has libraries for everything. Batteries are included. And it has highly optimized libraries for math and data manipulation. Finally, it is a glue language.</p>
<p>So, what is Python not good for? I would name three fields: UI (desktop applications), mobile development, and game development.
As you can see all my cases are related to end-users interaction with a Python application. In all these cases it's possible to develop good software with Python. But it's not worth doing it at least for production.</p>
<h2>User Interface</h2>
<p>What is the best language for UI? It depends on the platform. And in any case, Python isn't at the top of a list.
For web UI you have no choice. JavaScript is the only option. WebAssembly is at an early stage. And as far as I know, there is no Python support.</p>
<p>If we talk about desktop applications, the option for Mac is Swift, the option for Windows is C#. Linux? C/C++. Want something cross-platform? Use C/C++.</p>
<p>I don't say that there are no GUI libraries for Python. There are some. And they are OK if you are writing an application for yourself or other developers. They can spend a lot of time solving problems with it.</p>
<p>But regular users do not care much how close your program to platform guidelines. If it works reasonably well, they don't notice that you use non-native controls. But if they have to make efforts to install or update it, they'll find something else.
And it's a big problem for Python. It is hard to build one executable file with Python GUI application inside. And it's even more difficult to update it. I don't say it is impossible. It's not worth doing it in most cases.</p>
<h2>Mobile Development</h2>
<p>There is Kivy. It's good... for pet projects or for something that is not important for the business. Why? Because it is so fragile. Accessing camera or other device features usually depends on binary libraries that updates with different speed. And you'll get random errors after updates. Many extra libraries have no maintainers.</p>
<p>So for mobile development, the best option is a native application. The second option is Cordova or something similar. Depending on your business Cordova can be the best option. But in any case, I doubt that there is a place for Python in the list of options for mobile development.</p>
<h2>Game development</h2>
<p>The main problem with using Python for game development is not the luck of mature frameworks. There are some. However, they are not popular. They are not cross-platform. So that means your game will be for PC only forever. Other platforms? Develop a new game, but this time not using Python.</p>
<p>Marketing a game is incredibly hard. It is wise to find a publisher who can help you to promote your game. But again, the vast majority of publishers will not publish your games because of Python.</p>
<h2>Conclusion</h2>
<p>I'm using Python almost for everything. I even thinking about writing some music with it (there are some possibilities to do it). However, I will not start a new mobile application with Python at least that is supposed to be published in stores.
What do you think? What is Python not good for?</p>Python Typing Hints List2019-05-24T10:00:00+05:002019-05-24T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-05-24:/python-typing-hints-list.html<p>Here is my small list of sources of type hints in Python that I use almost every day.</p>
<p>Here is my small list of sources of type hints in Python that I use almost every day.</p>
<p>Recently I changed a project and a team. Now I'm working on DWH and ETL-processes. It is a quite big project. There is roughly the same amount of LOC but there are more developers working on it. I'm sure that type hints are more necessary for it.</p>
<p>Not only because of the number of developers but because of specific features of DWH/ETL-projects. They tend to get data from different sources and store them in different storages for different types of consumers. That leads to a code fragmentation. Without "the glue" the project falls apart to a bunch of scripts.</p>
<p>As soon as I joined the project I started to add type hints to important parts that worked on. Gradually other colleges joined in. After a while, we got a critical mass of typed Python code. That's why we decided to make type hinting obligatory (not for the entire project, but for a significant part of it).</p>
<p>I was asked to make a list of the most necessary and useful links that could help in everyday work. There were several requirements to that list:</p>
<ul>
<li>It should contain an as small amount of links as possible</li>
<li>It should contain only that is useful for everyday work</li>
<li>It should contain only links to short dense articles</li>
</ul>
<p>I tried to follow these restrictions not always successfully. However, I did my best. So here is my list.</p>
<ul>
<li><em><a href="https://realpython.com/python-type-checking/">The Ultimate Guide to Python Type Checking</a></em></li>
</ul>
<p>It is a very long Real Python's tutorial. It covers almost everything from type theory to mypy setup. It's really The Ultimate Guide to Python Type Checking. It's the best place to start. The main disadvantage of this source that it is too long.</p>
<ul>
<li><em><a href="https://mypy.readthedocs.io/en/latest/cheat_sheet.html">Type hints cheat sheet</a></em></li>
</ul>
<p>In contrast to the Ultimate Guide this cheat sheet is extremely short. It is easy to use. Mypy developers cover all basic cases (85-90% of all cases that I've ever met). At the very beginning of my type of hinting experiments, I used to use it constantly. However, this source covers only very basic cases. If you need to solve something complex, the cheat sheet isn't for you.</p>
<ul>
<li><em><a href="https://docs.python.org/3/library/typing.html">Support for type hints — Python documentation</a></em></li>
</ul>
<p>Python Documentation is one of the reasons for my love for this language. Docs for typing library isn't an exception. The majority of the questions can be solved here. Almost always it's enough just to look through it to get the right idea of type hinting pattern.</p>
<ul>
<li><em><a href="https://www.python.org/dev/peps/pep-0484/">PEP 484 (Type Hints)</a></em> </li>
<li><em><a href="https://www.python.org/dev/peps/pep-0483/">PEP 483 (The Theory of Type Hints)</a></em></li>
</ul>
<p>Sometimes documentation isn't enough. It's time for PEP 484 and PEP 483. I think these two PEPs are only I've ever used in everyday work. They are really handful to understand the logic on which type hinting is based. Without that understanding often it's next to impossible to find an appropriate solution on how to add a hint in a code.</p>
<p>That is all the links I use more or less regularly. Of course, sometimes I have to look through StackOverflow. But as usual, you have to know what you are looking for and how to use it. Sometimes the right variant isn't the most popular one or checked as "right" one.</p>
<p>All these links tell nothing about creating processes for type checking in a project. I wrote about it in 2017. Here is the <a href="https://www.alexkorablev.com/typing-ci.html">link to this article</a>.</p>Codingame: Clash of Code2019-04-26T10:00:00+05:002019-04-26T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-04-26:/clash-of-code.html<p>In January I <a href="https://www.alexkorablev.com/2019-plans.html">wrote</a> that I want to make a big and interesting project with <a href="https://www.codingame.com">Codingame</a> - develop an ML-based bot for LEGENDS OF CODE & MAGIC. Procrastinating an attempt to do it (again...) I found out that there is something really interesting on Codingame. <a href="https://www.codingame.com/multiplayer/clashofcode">Clash of Code!</a> It is a tiny coding competition. After playing them for a while I can say they are awesome to relax from daily routine.</p>
<p>In January I <a href="https://www.alexkorablev.com/2019-plans.html">wrote</a> that I want to make a big and interesting project with <a href="https://www.codingame.com">Codingame</a> - develop an ML-based bot for LEGENDS OF CODE & MAGIC. Procrastinating an attempt to do it (again...) I found out that there is something really interesting on Codingame. <a href="https://www.codingame.com/multiplayer/clashofcode">Clash of Code!</a> It is a tiny coding competition. After playing them for a while I can say they are awesome to relax from daily routine.</p>
<h2>Clash of Code Format</h2>
<p>These mini competitions are build basing on these rules:</p>
<ul>
<li>there is really simple coding/algorithmic challenge that can be sold in 5 minutes;</li>
<li>there is a 15-minute-long limit;</li>
<li>there is one randomly selected winning condition:<ul>
<li>solve the fast as possible;</li>
<li>write the shortest code;</li>
<li>guess what to do by observing a provided set of tests.</li>
</ul>
</li>
</ul>
<p>These additional rules make challenge not so easy at all. My favorite rule is to write the shortest solution. You have to revise all your knowledge of the standard library and refresh all Python hacks.</p>
<h2>What's the purpose of it?</h2>
<p>I suppose Cash of Code has only one positive outcome. It is great "refresher". There are no more practical results in these challenges.</p>
<p>On the other hand, switch to something else that strictly time-limited can be great. Relax for 15 minutes and return to work. Facebook doesn't work this way, does it?</p>
<p>I think it worth trying. It gives a really interesting experience. And I promise to start the bot project before last proton decays. I swear! :)</p>Using import foo.bar.baz as fbb2019-03-18T10:00:00+05:002019-03-18T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-03-18:/import-aliases.html<p>There is a wonderful feature in Python import statement to make an alias for an imported object (<code>import foo.bar.baz as fbb</code>). It allows to avoid name conflicts and improve code readability if necessary. And of course, using it you can easily turn your perfect code into a completely unreadable mess. Recently I’ve seen this several times. I’ve met some strange abbreviations and confusing aliases.</p>
<p>There is a wonderful feature in Python import statement to make an alias for an imported object (<code>import foo.bar.baz as fbb</code>). It allows to avoid name conflicts and improve code readability if necessary. And of course, using it you can easily turn your perfect code into a completely unreadable mess. Recently I’ve seen this several times. I’ve met some strange abbreviations and confusing aliases.</p>
<p>Let me show you an example of a problem with abbreviations. Here is a good example. It’s not from real code, but it looks very similar.</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">facebook</span> <span class="kn">import</span> <span class="n">utils</span> <span class="k">as</span> <span class="n">fbu</span>
<span class="kn">from</span> <span class="nn">twitter</span> <span class="kn">import</span> <span class="n">units</span> <span class="k">as</span> <span class="n">tu</span>
</code></pre></div>
<p>Ok. We resolve the name conflict. But what does <code>tu</code> mean? If I see such names in code, I have to make extra efforts to understand what an author means. It is much better if there is something like this instead of ugly abbreviations:</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">facebook</span> <span class="kn">import</span> <span class="n">utils</span> <span class="k">as</span> <span class="n">facebook_utils</span>
<span class="kn">from</span> <span class="nn">twitter</span> <span class="kn">import</span> <span class="n">units</span> <span class="k">as</span> <span class="n">twitter_utils</span>
</code></pre></div>
<p>Now the code is much cleaner. Sure variable names are longer. But who spares bytes nowadays? I think I would use this formula even if there was only one import from utils library, just to improve readability.</p>
<p>It looks obvious to me. However, it is not so easy to prove that readability is important. Usually, other developers answer that you can always check the import section to understand their meanings. Of course, I can. There are no problems in any good IDE: point to an interesting position in code, do to a declaration, read it and go back.</p>
<p>But developers read code mostly on Github or similar systems. We usually read small diff, that may not include an import section. And you have to “unfold” that diff to find the imports. And there is no code navigation. So let’s save the time of readers using meaningful aliases in imports.</p>
<p>What do you think? Is it worth rejecting PRs?</p>Please Do Not Use "else"2019-01-29T10:00:00+05:002019-01-29T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-01-29:/if-else.html<p>I don’t like <code>else</code> if there is return-statement in both branches. It is always possible to write code without that unnecessary <code>if</code> appendage. Such code looks more accurate, more clear and it is easier to read. Why so many developers still use this useless <code>else</code>?</p>
<p>I don’t like <code>else</code> if there is return-statement in both branches. It is always possible to write code without that unnecessary <code>if</code> appendage. Such code looks more accurate, more clear and it is easier to read. Why so many developers still use this useless <code>else</code>?</p>
<p>Reviewing my colleagues’ code I noticed that he can improve it omitting <code>else</code>. In response, he gave me a link to PEP8 where <code>if-else</code> with <code>return</code> noticed as a possible variant. His code looks like this:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_magic_number</span><span class="p">(</span><span class="n">use_magic</span><span class="p">):</span>
<span class="k">if</span> <span class="n">use_magic</span><span class="p">:</span>
<span class="k">return</span> <span class="n">calculate_with_magic</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">calculate_with_science</span><span class="p">()</span>
</code></pre></div>
<p>For a machine this code is perfect. It does what it should do. But I, as human, care about readability. And I always, in this case, stop reading and ask myself:</p>
<ul>
<li>What does this function return?</li>
<li>Are both return-statements reachable with any input?</li>
</ul>
<p>Even such short functions manage me to stop and think. But not always they are so simple and short.</p>
<p>So I have a question, why we can’t omit <code>else</code>? I think profit is obvious:</p>
<ul>
<li>Code becomes one-line shorter.</li>
<li>Less intend for one of the branches.</li>
<li>We get a default return-statemen for a function.</li>
</ul>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_magic_number</span><span class="p">(</span><span class="n">use_magic</span><span class="p">):</span>
<span class="k">if</span> <span class="n">use_magic</span><span class="p">:</span>
<span class="k">return</span> <span class="n">calculate_with_magic</span><span class="p">()</span>
<span class="k">return</span> <span class="n">calculate_with_science</span><span class="p">()</span>
</code></pre></div>
<p>Is it more readable? Yes, it is. Is there any difference for an interpreter? No, it isn’t. So why do we use this useless construction where it is not necessary? PEP8 and I-will-need-it-later are not the arguments.</p>
<p>What do you think? Is this all just matter of taste?</p>Plans for 20192019-01-02T10:00:00+05:002019-01-02T10:00:00+05:00akorablevtag:www.alexkorablev.com,2019-01-02:/2019-plans.html<p>Today is 2nd of January 2019. It is a time to make some plans for this blog. I want to write at least 12 blog posts. There are two topics that I find very interesting: CodinGames and Kaggle challenges.</p>
<p>Today is 2nd of January 2019. It is a time to make some plans for this blog. I want to write at least 12 blog posts. There are two topics that I find very interesting: CodinGames and Kaggle challenges.</p>
<h3>CodinGames</h3>
<p>Several posts will be about <a href="https://www.codingame.com/multiplayer/bot-programming/legends-of-code-magic">LEGENDS OF CODE & MAGIC</a> challenge. I choose this it because I love board and card games. I used to play Magic the Gathering several years ago. It would be interesting to build a bot for an MtG-like game. I have an idea to make something like AlphaGo or at least use a similar approach to get into Legendary league in this challenge.</p>
<h3>Kaggle</h3>
<p>Another interesting topic is Kaggle. I want to improve my ML skills. But I haven’t many things to do with ML at work. That’s why I decided to spend some time taking part in challenges hosted on Kaggle. To keep me motivated I set up a goal to get a Kaggle Master.</p>
<p>The main reason for that is to learn the ropes. My knowledge of the topic is more theoretical than practical. I want to change it. Because I have only $20-25 monthly for servers I am going to take part in text-based challenges only.</p>
<h3>Other topics</h3>
<p>Besides Kaggle and CodinGames I will write book reviews and work-related posts if something interesting happens. Also, I have a bunch of things to automate. Maybe I’ll write a couple of posts about it.</p>
<p>So that is my plans for 2019 posts. Happy New Year!</p>Mobile App with Kivy and Python? Mmm… Not now2018-10-25T22:00:00+05:002018-10-25T22:00:00+05:00akorablevtag:www.alexkorablev.com,2018-10-25:/kivy-mobile.html<p>There are several GUI frameworks for Python. Most of them are for desktop applications. Kivy is an exception. Using it you can build project for mobile platforms. Unfortunately, it is not production ready. You have to juggle versions of libraries to make it works. So even having a t-shirt that says <code>print("Python is my favorite language")</code> I have to accept that Python is not for mobile development.</p>
<p>There are several GUI frameworks for Python. Most of them are for desktop applications. Kivy is an exception. Using it you can build project for mobile platforms. Unfortunately, it is not production ready. You have to juggle versions of libraries to make it works. So even having a t-shirt that says <code>print("Python is my favorite language")</code> I have to accept that Python is not for mobile development.</p>
<p>Ok. You have to do the same things for web applications. But in this case, you have to manage python dependencies only (in 99% of cases) or you can control your environment. Basically, you can build a docker image as you want with any libraries you need. But Kivy brings a lot of binary dependencies that should be installed using platform-specific external tools. And now you can’t control the environment. And this changes everything. </p>
<p>Recently I decided to build a small QR-code reading app for Android. I’m using Python every day, so I had no reasons not to use Kivy. And a couple of hours later I had a desktop application that worked quite well.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">kivy.app</span> <span class="kn">import</span> <span class="n">App</span>
<span class="kn">from</span> <span class="nn">kivy.lang</span> <span class="kn">import</span> <span class="n">Builder</span>
<span class="kn">from</span> <span class="nn">zbarcam.zbarcam</span> <span class="kn">import</span> <span class="n">zbarcam</span>
<span class="n">DEMO_APP_KV_LANG</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2">BoxLayout:</span>
<span class="s2"> orientation: 'vertical'</span>
<span class="s2"> ZBarCam:</span>
<span class="s2"> id: zbarcam</span>
<span class="s2"> code_types: 'qrcode', 'ean13'</span>
<span class="s2"> Label:</span>
<span class="s2"> size_hint: None, None</span>
<span class="s2"> size: self.texture_size[0], 50</span>
<span class="s2"> text: 'test: ' + ', '.join([str(symbol.data) for symbol in zbarcam.symbols])</span>
<span class="s2">"""</span>
<span class="k">class</span> <span class="nc">DemoApp</span><span class="p">(</span><span class="n">App</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Builder</span><span class="o">.</span><span class="n">load_string</span><span class="p">(</span><span class="n">DEMO_APP_KV_LANG</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">DemoApp</span><span class="p">()</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>Actually, I spent 5 minutes to find and copy-paste this snippet and 1 hour 55 minutes to make it work. I supposed that it should take 10-15 minutes maximum. Because of the problem with a binary dependency, zbarcam from kivy-garden was too old and didn't work. And then I’ve spent 2 days trying to build an android project. Without any success. I failed to find working pair versions for Kivy and Cython. </p>
<p>I decided to stop there. I love Python, but there are more stable technologies to make the thing done. Event Cordova and JS suit this task much better. They are more mature, and stable. Xamarin another good option. Do not use technology in a field where it doesn’t work just because you know it.</p>Optional or Not in Abstract Classes2018-08-27T10:00:00+05:002018-08-27T10:00:00+05:00akorablevtag:www.alexkorablev.com,2018-08-27:/optional-or-not.html<p>Type hints are not mandatory in Python. So you can use them or not. If you decide work with them you’ll face many difficulties annotating functions or variables. This article shows my point of view on one specific case.</p>
<p>Type hints are not mandatory in Python. So you can use them or not. If you decide work with them you’ll face many difficulties annotating functions or variables. This article shows my point of view on one specific case.</p>
<p>For instance, we have a class hierarchy that looks like one in the code below. It could be classes for database mapping or commands. The most important part of this example is that there are one abstract class and several real classes. And it doesn’t matter whether the abstract class is annotated as abstract or not. I suppose that it is used as an abstract one.</p>
<div class="highlight"><pre><span></span><code> <span class="k">class</span> <span class="nc">A</span><span class="p">:</span>
<span class="n">arrt</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">a</span><span class="p">):</span>
<span class="n">arrt</span> <span class="o">=</span> <span class="s1">'B'</span>
</code></pre></div>
<p>As you can see there is one class attribute that must have string value in each real class. But there is no special value for abstract classes and you can’t use empty string there.
In this case, you have two options if you want to add type hint: use Optional or not. Let’s discuss the first option.</p>
<div class="highlight"><pre><span></span><code> <span class="k">class</span> <span class="nc">A</span><span class="p">:</span>
<span class="n">arrt</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># type: Optional[str]</span>
</code></pre></div>
<p>It looks good. But there is one problem. This hint suppose that <code>arrt</code> could be either <code>str</code> or <code>None</code> . So this class is OK:</p>
<div class="highlight"><pre><span></span><code> <span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">attr</span> <span class="o">=</span> <span class="kc">None</span>
</code></pre></div>
<p>However, it isn’t exactly what we want. Our goal is to prevent this.
Second option without <code>Option</code> works much better. </p>
<div class="highlight"><pre><span></span><code> <span class="k">class</span> <span class="nc">A</span><span class="p">:</span>
<span class="n">arrt</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># type: str</span>
</code></pre></div>
<p>In this case class <code>C</code> will raise mypy error. And this is exactly what we want to achieve.
Type hinting is an additional thing in Python. Sometimes dynamic nature of the language and a legacy code cause problems. <code>Optional</code> is a good choice sometimes but not always. </p>Coders Strike Back Post-mortem2018-07-12T10:00:00+05:002018-07-12T10:00:00+05:00akorablevtag:www.alexkorablev.com,2018-07-12:/coders-strike-back.html<p>The latest couple of months I've spent around 40 hours at weekends and in evenings trying to write a bot that is able to finish the track of <a href="https://www.codingame.com/multiplayer/bot-programming/coders-strike-back">Coders Strike Back</a> faster than opponents. During this time I'd made quite a lot of mistakes before I got into the Legendary league. This is a post-mortem for that project.</p>
<p>The latest couple of months I've spent around 40 hours at weekends and in evenings trying to write a bot that is able to finish the track of <a href="https://www.codingame.com/multiplayer/bot-programming/coders-strike-back">Coders Strike Back</a> faster than opponents. During this time I'd made quite a lot of mistakes before I got into the Legendary league. This is a post-mortem for that project.</p>
<p>I have an important disclaimer. Almost all bots that I wrote was in Python. However, I passed high leagues with the C++ bot. I don't think that there is something wrong with Python. I think there were several errors in my code that I decided not to fix. I'm a lazy-ass.</p>
<h2>CodinGames</h2>
<p>At first, I have to say a couple of words about the website there competitions is hosted. CodinGames is a collection of programming challenges that are presented as games. Some of them are puzzles. Sometimes you have to beat an AI. In others, you compete with bots written by other users. There are many challenges and puzzles with different difficulty and that covers different fields. They all have one thing in common. You have to guide your hero or heroes by writing a bot.</p>
<p>In any case, all games on this website are masked programming challenges. I think a big fraction of them you can find on websites like HackerRank in more direct form. This is the main advantage of CodinGames. It is much more interesting to spend time trying to understand why your bot is so slow than why your solution fails test 16 and gets a timeout in test 20.</p>
<p>I think CodinGames and HackerRank can be used for different things. HackerRank is a great interview preparation tool. It is extremely useful when you want to learn how to solve a standard algorithmic challenge that is formulated as they are usually formulated for the technical interviews. Despite not having any time limits on HackerRank, I often have a filling that someone urges me saying "You have an hour to solve it".</p>
<p>CodinGames is more like a laboratory. Challenges here are more about finding new ways to solve them. I have not so big experience working with this site, but my internal timer hasn't started ever. As a result, I spent much more time finding a solution that I planned. Challenges in form of games give me a feeling of freedom. On this site much easier to say yourself that you are stuck and it's time to google. On one hand, I want to solve everything myself without any help. On the other hand, this freedom cases that I try more approaches to solve a problem then I've ever done solving any HackerRank challenge.</p>
<h2>Coders Strike Back</h2>
<p>Coders Strike Back is a good illustration for that difference between HackerRank and CodinGames. In spite of challenges on HackerRank, there is no the right way to write a code for that challenge. During a search of an inspiration, I found that good result could be achieved using a variety of methods form some heuristics algorithms to machine learning. </p>
<p>In this game, your goal is to flight thought checkpoints several laps. I must to it faster than your opponent. The competition is divided into leagues. You'll pass from one league to other if your AI gets more points in races with other players' bots than league boss' AI.</p>
<p>There higher league the more rules are in it: boost, shield, second pot... Bosses became better and faster. To be honest it is easy to write a bot based on heuristics for leagues below golden league. On this stages, there is no sense to write anything more complex than if-based AI. The most valuable tip here is to forget about opponents and to concentrate on improving accuracy and speed. It is not so hard to do. </p>
<h2>Golden league</h2>
<p>The golden league is the first league with serious resistance. For a long time, I couldn't get more than a half of the points that league boss got. With my naive approach, I had a place near 2000 or so only because there were a lot of broken bots that was able to make 3 or 4 turns before timeouts.</p>
<p>I hadn't wanted to use complex algorithms and had planned to pass to the legendary league with a bot based on heuristics only. My mood was getting worse as my code becoming more and more complex. Then after a couple hours of intensive googling, I found an extremely interesting and simple heuristics. The goal coordinate equals to the coordinate of the next checkpoint minus tripled velocity projection.</p>
<p>So as I got a very compact bot that however performed very well finishing at places around 300.</p>
<div class="highlight"><pre><span></span><code><span class="n">laps</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">input</span><span class="p">())</span>
<span class="n">cp_count</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">input</span><span class="p">())</span>
<span class="n">checkpoints</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">cp_count</span><span class="p">):</span>
<span class="n">checkpoint_x</span><span class="p">,</span> <span class="n">checkpoint_y</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">j</span><span class="p">)</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">input</span><span class="p">()</span><span class="o">.</span><span class="n">split</span><span class="p">()]</span>
<span class="n">checkpoints</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">checkpoint_x</span><span class="p">,</span> <span class="n">checkpoint_y</span><span class="p">))</span>
<span class="k">while</span><span class="p">(</span><span class="mi">1</span><span class="p">):</span>
<span class="n">fp_x</span><span class="p">,</span> <span class="n">fp_y</span><span class="p">,</span> <span class="n">fp_vx</span><span class="p">,</span> <span class="n">fp_vy</span><span class="p">,</span> <span class="n">fp_angle</span><span class="p">,</span> <span class="n">fp_next_cp_id</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">j</span><span class="p">)</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">input</span><span class="p">()</span><span class="o">.</span><span class="n">split</span><span class="p">()]</span>
<span class="n">sp_x</span><span class="p">,</span> <span class="n">sp_y</span><span class="p">,</span> <span class="n">sp_vx</span><span class="p">,</span> <span class="n">sp_vy</span><span class="p">,</span> <span class="n">sp_angle</span><span class="p">,</span> <span class="n">sp_next_cp_id</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">j</span><span class="p">)</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">input</span><span class="p">()</span><span class="o">.</span><span class="n">split</span><span class="p">()]</span>
<span class="nb">print</span><span class="p">(</span>
<span class="s2">"</span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">checkpoints</span><span class="p">[</span><span class="n">fp_next_cp_id</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="mi">3</span><span class="o">*</span><span class="n">fp_vx</span><span class="p">,</span>
<span class="n">checkpoints</span><span class="p">[</span><span class="n">fp_next_cp_id</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">3</span><span class="o">*</span><span class="n">fp_vy</span><span class="p">,</span>
<span class="s2">"BOOST"</span>
<span class="p">)</span>
<span class="p">)</span>
<span class="nb">print</span><span class="p">(</span>
<span class="s2">"</span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">checkpoints</span><span class="p">[</span><span class="n">sp_next_cp_id</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="mi">3</span><span class="o">*</span><span class="n">sp_vx</span><span class="p">,</span>
<span class="n">checkpoints</span><span class="p">[</span><span class="n">sp_next_cp_id</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">3</span><span class="o">*</span><span class="n">sp_vy</span><span class="p">,</span>
<span class="s2">"BOOST"</span>
<span class="p">)</span>
<span class="p">)</span>
<span class="nb">input</span><span class="p">()</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
<span class="nb">input</span><span class="p">()</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</code></pre></div>
<p>And again, adding new logic made it worse. I needed a new better more advanced approach. Then I found several articles with one interesting idea. If you can build a number of possible future trajectories for your pot, evaluating them somehow you will find the optimal solution.</p>
<ul>
<li><a href="https://www.codingame.com/blog/genetic-algorithms-coders-strike-back-game/">GENETIC ALGORITHMS IN CODERS STRIKE BACK GAME</a></li>
<li><a href="https://www.codingame.com/blog/coders-strike-back-pb4608s-ai-rank-3rd/">CODERS STRIKE BACK – PB4608’S AI (RANK: 3RD)</a></li>
<li><a href="http://files.magusgeek.com/csb/csb_en.html">Coders Strikes Back</a></li>
</ul>
<p>In addition to that plans generation, it is rather easy to do, you need to build a model that can predict pot positions after each turn. The latest link in the list describes this model and gives detailed instruction on how to implement it on your own.</p>
<p>Unfortunately, straightforward Python realization of it is quite slow. My best result is 60 plans without risk of a timeout. I guess it is possible to improve it using NumPy. But I decided to take an easy way. By that time I had been quite tired of this project and I thought only how to finish it. So I returned to Google and tried to find an appropriate template. </p>
<p>Such template was found in 20 minutes. It was <a href="https://github.com/Inoryy/csb-ai-starter">a piece of code</a> written by Roman Ring. And after changing evaluating function a little bit I was promoted to Legendary League. </p>
<h2>Legendary League / Conclusion</h2>
<p>Changing only speed limit I got 40th place in the league. Because promotion to the Legendary League was the project goal, I stopped. However, having a good moving prediction model template you can get better results changing some parts, evaluation function and plan generation algorithm for example.</p>4 Articles about Python Internals2018-03-29T10:00:00+05:002018-03-29T10:00:00+05:00akorablevtag:www.alexkorablev.com,2018-03-29:/python-internals.html<p>Maybe, knowing Python internals is not a thing you should know to be a good Python developer. However, if you want to improve your code it becomes more important. If you know it works under the hood, you write a code with less stupid mistakes and architecture issues.</p>
<p>Maybe, knowing Python internals is not a thing you should know to be a good Python developer. However, if you want to improve your code it becomes more important. If you know it works under the hood, you write a code with less stupid mistakes and architecture issues.</p>
<h3>#1 <a href="http://pgbovine.net/cpython-internals.htm">CPython internals: A ten-hour codewalk through the Python interpreter source code</a></h3>
<blockquote>
<p>Here are nine lectures walking through the internals of CPython, the canonical Python interpreter implemented in C. They were from a dynamic programming languages course that I taught in Fall 2014 at the University of Rochester. The format isn't ideal, but I haven't seen this level of detail about CPython presented online, so I wanted to share these videos.</p>
</blockquote>
<h3>#2 <a href="https://dawranliou.com/getting-started-python-internals.html">Getting Started with Python Internals</a></h3>
<p>Getting Started with Python Internals is a summary of Philip Guo's codewalk. The main difference is that in this article Python 3.6 is used.</p>
<h3>#3 <a href="https://eli.thegreenplace.net/2010/06/30/python-internals-adding-a-new-statement-to-python/">Python internals: adding a new statement to Python</a></h3>
<blockquote>
<p>This article is an attempt to better understand how the front-end of Python works. Just reading documentation and source code may be a bit boring, so I'm taking a hands-on approach here: I'm going to add an until statement to Python.</p>
</blockquote>
<p>Eli Bendersky has written many interesting articles about Python internals. You can find them <a href="https://eli.thegreenplace.net/tag/python-internals">here</a></p>
<h3>#4 <a href="https://rushter.com/blog/python-garbage-collector/">Garbage collection in Python: things you need to know</a></h3>
<blockquote>
<p>Usually, you do not need to worry about memory management when the objects are no longer needed Python automatically reclaims the memory from them. However, understanding how GC works can help you write better Python programs.</p>
</blockquote>
<p>Like Eli Bendersky, Artem Golubin has written <a href="https://rushter.com/blog/tags/cpython/">several articles about CPython</a> that worth reading.</p>
<h3>Extra...</h3>
<h4>Pypy:</h4>
<ul>
<li>http://doc.pypy.org/en/latest/interpreter.html</li>
<li>http://doc.pypy.org/en/latest/cpython_differences.html</li>
<li>https://morepypy.blogspot.ru/2017/10/how-to-make-your-code-80-times-faster.html</li>
</ul>
<h4>Another list:</h4>
<ul>
<li>https://github.com/amygdalama/python-internals </li>
</ul>Computer Science Distilled2018-02-28T10:00:00+05:002018-02-28T10:00:00+05:00akorablevtag:www.alexkorablev.com,2018-02-28:/computer-science-distilled.html<p>Are you looking for a simple thin Computer Science book? Maybe, <a href="https://sourcemaking.com/computer-science-distilled">Computer Science Distilled</a> is for you. Unfortunately only if you have no plans to code for living. In this case, it is a good introduction to computer science world. If you are professional developer or CS student, skip this book.</p>
<p>Are you looking for a simple thin Computer Science book? Maybe, <a href="https://sourcemaking.com/computer-science-distilled">Computer Science Distilled</a> is for you. Unfortunately only if you have no plans to code for living. In this case, it is a good introduction to computer science world. If you are professional developer or CS student, skip this book.</p>
<blockquote>
<p>It’s a short book, written in plain, basic English. It presents ideas in their simplest form. It focuses on practical aspects of computer science that matter most: everyday things that directly impact the quality of your code.</p>
</blockquote>
<p>This book consists of 8 chapters that cover every aspect of computer science. Do you think it has thousands of pages? No, it doesn't. There are only 170 pages. So how it can cover all aspects of CS? It can't. Actually, this book contains an extremely short introduction into some CS fields.</p>
<p>Computer Science Distilled is very similar to physics books for amateurs. In such books, there are no formulas, no math, only text explanations in plain English. The main purpose of that kind of books is to entertain readers.</p>
<p>I think the main problem with the book is that the author skips detailed explanation of algorithms. Sometimes he gives you a graphical explanation only.</p>
<p>Anyway, this book has its own audience but, I hope, not from professional developers. </p>Tuple[Callable, Any, ...]2018-01-29T10:00:00+05:002018-01-29T10:00:00+05:00akorablevtag:www.alexkorablev.com,2018-01-29:/tuple-type.html<p>Type hints could help you a lot with a big Python project. However, they sometimes require code refactoring. I wrote about it last year in <a href="https://www.alexkorablev.com/typing-ci.html">this article</a>, but I have found a good example for this only now.</p>
<p>Type hints could help you a lot with a big Python project. However, they sometimes require code refactoring. I wrote about it last year in <a href="https://www.alexkorablev.com/typing-ci.html">this article</a>, but I have found a good example for this only now.</p>
<p>Look at this code:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">workflow_step</span><span class="p">):</span>
<span class="n">func</span> <span class="o">=</span> <span class="n">step</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">args</span> <span class="o">=</span> <span class="n">step</span><span class="p">[</span><span class="mi">1</span><span class="p">:]</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
</code></pre></div>
<p>What can we get from this piece of code? That <code>workflow_step</code> is a sequence witch first element is a <code>Callable</code> object and the other items, if any, are its arguments. Let's assume that <code>workflow_step</code> is a <code>Tuple</code>. The problem is that the type <code>(Tuple) -> Any</code> for this function is utterly uninformative. Can I call process with an empty tuple? Is <code>(1, 2, 3)</code> OK?</p>
<p><code>(Tuple[Callable, Any, ...]) -> Any</code> is much better. However, it is not valid. Typing library doesn't allow such syntax. For simple cases, one of the options is to use <code>Union[Tuple[Callable], Tuple[Callable, Any]]</code> or something similar. In any other cases there is the only one option: rewrite this piece of code.</p>
<p>There are plenty of options how to do it, for example, using <code>NamedTuple</code>.</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">WorkflowStep</span><span class="p">(</span><span class="n">NamedTuple</span><span class="p">):</span>
<span class="n">callable</span> <span class="c1"># type: Callable</span>
<span class="n">args</span> <span class="c1"># type: Tuple</span>
<span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">workflow_step</span><span class="p">):</span>
<span class="c1"># type: (WorkflowStep) -> Any</span>
<span class="k">return</span> <span class="n">step</span><span class="o">.</span><span class="n">callable</span><span class="p">(</span><span class="n">args</span><span class="o">.</span> <span class="n">args</span><span class="p">)</span>
</code></pre></div>
<p>Maybe this is not the best piece of code you've ever seen. But no doubts it is much more structured and readable than it was before. </p>How to add type hints into Python 2.7 project2017-12-11T10:00:00+05:002017-12-11T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-12-11:/typing-ci.html<p>Many times I wrote that type hints in Python help to work with big or moderate projects. However, if you decide to add them to your project, you have to check your project regularly using CI. And this kind of checks is not easy to implement. This article is my story about obstacles in this process. </p>
<p>I'm trying to add type hints to our project for a while. And now I have a bunch of methods how to do it more efficiently, fast, and painless. I spend many days trying to find them. Now I want to share my experience.</p>
<p>Many times I wrote that type hints in Python help to work with big or moderate projects. However, if you decide to add them to your project, you have to check your project regularly using CI. And this kind of checks is not easy to implement. This article is my story about obstacles in this process. </p>
<p>I'm trying to add type hints to our project for a while. And now I have a bunch of methods how to do it more efficiently, fast, and painless. I spend many days trying to find them. Now I want to share my experience.</p>
<p>I have to notice that the most painful problems are connected with Python 2.7 syntax. Many of them vanish after switching to Python 3.x. Another big problem is lack of good examples. It easy to find some good code examples in our codebase. However, all of them are under the NDA and they are relatively big. On the other hand, small examples are not enough illustrative.</p>
<h2>Small pieces</h2>
<p>Do not try to add type hints for the whole project. Firstly, if you can add hints for the whole project in a reasonable time, possibly your project isn't big enough. Secondly, if your project is big, you will dramatically slow your down. You will get a zillion of errors.</p>
<p>It is much better to divide a project into smaller pieces and add types to them. Ideally, that peace should be small enough to fix all errors in a half an hour or so. For me, it means that mypy should return no more than 10-15 errors. So I usually check one middle size module a time. There is no anything wrong if you slice the project into thin pieces.</p>
<p>It is better to fix errors bottom-up way. I check submodules first. After them, I check their parent module.</p>
<h2>Make type checks for every pull request you made</h2>
<p>It's very easy to check types for a couple of files. It's a bit harder to do the same for a dozen. Usually, pull request consist of less than dozen files. So checking types for that files doesn't take you much time. However, it could increase a velocity of adding type annotation into your project. </p>
<p>This way have a bonus. It could show you several potential problems in the project. It is usually hard to understand where to start adding type checks first. A pull request shows errors that should be fixed first. Files from PRs are good candidates to check. They are under active development.</p>
<h2>Weak checks first</h2>
<p>It's cool to add type hints with maximum details. It's cool to have code without any <code>Any</code> annotations. However, it's next to impossible to do from scratch. Some parts could require refactoring. Sometimes there is no way to pass all checks with a current version of the code. Then you have two options: rewrite that parts or ignore checks.</p>
<p>Meanwhile, weak checks are better than no checks. General type hints are better than no hints. In most cases marking a parameter or a variable with the most generic type is more useful than a variable without any type hint. </p>
<p><code>Any</code> is a good option sometimes. At least it allows you to add some annotations. And mypy allows you to find all <code>Any</code> in your project later. So there is no reason to spend much time trying to find an accurate type for every line in a code. Make your project pass all weak checks first.</p>
<h2>CI first</h2>
<p>One of my worst mistakes was adding type hints before implementing automatic tests in CI. mypy shows a significant amount of errors even on a project without any type hints. Most of them are something like "Need type annotation for a variable" or error shows that some variable has different value type then it has in a base class.</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Foo</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="n">boo</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">class</span> <span class="nc">Boo</span><span class="p">(</span><span class="n">Foo</span><span class="p">):</span>
<span class="n">boo</span> <span class="o">=</span> <span class="s1">'boo'</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">example01</span><span class="p">.</span><span class="nl">py:</span><span class="mh">6</span><span class="o">:</span><span class="w"> </span><span class="nl">error:</span><span class="w"> </span><span class="n">Incompatible</span><span class="w"> </span><span class="n">types</span><span class="w"> </span><span class="n">in</span><span class="w"> </span><span class="n">assignment</span><span class="w"></span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">boo</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">foo</span> <span class="o">=</span> <span class="p">{}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">example02</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="w"> </span><span class="n">error</span><span class="p">:</span><span class="w"> </span><span class="n">Need</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="n">annotation</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">variable</span><span class="w"></span>
<span class="n">example02</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">3</span><span class="p">:</span><span class="w"> </span><span class="n">error</span><span class="p">:</span><span class="w"> </span><span class="n">Need</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="n">annotation</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">variable</span><span class="w"></span>
</code></pre></div>
<p>Usually, there are a lot of places in a project with that kind of errors. But there are no problems to fix them. It'll take no more than a couple of days to fix them all.</p>
<p>But if you have type annotations and no CI checks, errors could be more difficult to solve.</p>
<p><strong>Firstly, sometimes function interface does not match type annotations.</strong></p>
<p>This type of errors is common in Python 2.7 projects where comments are used for type hints. And such errors are not easiest to deal with. Reading function interface is not enough to understand types of its parameters.</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">datetime</span>
<span class="k">def</span> <span class="nf">convert_to_timedelta</span><span class="p">(</span><span class="n">step</span><span class="p">):</span>
<span class="c1"># does something and return timedelta</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">since</span><span class="p">,</span> <span class="n">until</span><span class="p">,</span> <span class="n">step</span><span class="p">):</span>
<span class="c1"># type: (datetime, datetime) -> Generator[datetime, None, None]</span>
<span class="n">step</span> <span class="o">=</span> <span class="n">convert_to_timedelta</span><span class="p">(</span><span class="n">step</span><span class="p">)</span>
<span class="k">while</span> <span class="n">since</span> <span class="o"><</span> <span class="n">until</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">since</span>
<span class="n">since</span> <span class="o">+=</span> <span class="n">step</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code>example03.py:8: error: Type signature has too few arguments
</code></pre></div>
<p>What type should be <code>step</code> in this context? Could it be <code>timedelta</code>? Perhaps. Or is it <code>int</code>? Why not?. It depends on <code>convert_to_timedelta</code> realisation. But there is no type hint for this function. So, fix this error is not easy.</p>
<p>Of course, you always can just delete hint for <code>foo</code> or choose <code>Any</code> as a type of <code>step</code>. It looks like a step backward. In some really difficult situation, it is an option.</p>
<p>If an unannotated parameter appears somewhere in the middle of function interface, it becomes even more difficult. For example, let's use the same function. But this time <code>step</code> appears in the middle.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">since</span><span class="p">,</span> <span class="n">step</span><span class="p">,</span> <span class="n">until</span><span class="p">):</span>
<span class="c1"># type: (datetime, datetime) -> Generator[datetime, None, None]</span>
<span class="n">step</span> <span class="o">=</span> <span class="n">convert_to_timedelta</span><span class="p">(</span><span class="n">step</span><span class="p">)</span>
<span class="k">while</span> <span class="n">since</span> <span class="o"><</span> <span class="n">until</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">since</span>
<span class="n">since</span> <span class="o">+=</span> <span class="n">step</span>
</code></pre></div>
<p>Let's assume that there was a good reason to put <code>step</code> parameter in the middle. It is just an example. Could <code>step</code> be a <code>datetime</code> or not this time? In general, it could. So <code>(datetime, datetime, datetime) -> datetime</code> is a good type for this function. However, looking through the function <code>foo</code> I doubt that.</p>
<p>It could be much harder to deal with a real function that is more longer and has more external calls. In this case, the only thing you can do is to analyze source code and all calls to find a glue.</p>
<p><strong>Secondly, without automatic checks there is no way how to understand if type hints are OK.</strong></p>
<p>Ok. Now we have this function:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">since</span><span class="p">,</span> <span class="n">until</span><span class="p">,</span> <span class="n">step</span><span class="p">):</span>
<span class="c1"># type: (datetime, datetime, int) -> Generator[datetime, None, None]</span>
<span class="n">step</span> <span class="o">=</span> <span class="n">convert_to_timedelta</span><span class="p">(</span><span class="n">step</span><span class="p">)</span>
<span class="k">while</span> <span class="n">since</span> <span class="o"><</span> <span class="n">until</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">since</span>
<span class="n">since</span> <span class="o">+=</span> <span class="n">step</span>
</code></pre></div>
<p>Later, we decide that <code>int</code> isn't a proper type for <code>step</code>, <code>timedelta</code> fits better. In all calls, we carefully change <code>int</code> to <code>timedelta</code>. Also, we've thrown away a line with <code>int</code> to <code>timedelta</code> conversion from the function. But how it usually happens, we forgot to change its type hint:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">since</span><span class="p">,</span> <span class="n">until</span><span class="p">,</span> <span class="n">step</span><span class="p">):</span>
<span class="c1"># type: (datetime, datetime, int) -> Generator[datetime, None, None]</span>
<span class="k">while</span> <span class="n">since</span> <span class="o"><</span> <span class="n">until</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">since</span>
<span class="n">since</span> <span class="o">+=</span> <span class="n">step</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">example05</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">9</span><span class="p">:</span><span class="w"> </span><span class="n">error</span><span class="p">:</span><span class="w"> </span><span class="n">Unsupported</span><span class="w"> </span><span class="n">operand</span><span class="w"> </span><span class="n">types</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="p">(</span><span class="s2">"datetime"</span><span class="w"> </span><span class="ow">and</span><span class="w"> </span><span class="s2">"int"</span><span class="p">)</span><span class="w"></span>
<span class="n">example05</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">9</span><span class="p">:</span><span class="w"> </span><span class="n">error</span><span class="p">:</span><span class="w"> </span><span class="n">Incompatible</span><span class="w"> </span><span class="n">types</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">assignment</span><span class="w"> </span><span class="p">(</span><span class="n">expression</span><span class="w"> </span><span class="n">has</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="s2">"int"</span><span class="p">,</span><span class="w"> </span><span class="n">variable</span><span class="w"> </span><span class="n">has</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="s2">"datetime"</span><span class="p">)</span><span class="w"></span>
</code></pre></div>
<p>Even in this tiny example <code>mypy</code> gives wrong recomendations. <code>datetime</code> is wrong type for <code>step</code>. The right variant is that:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">since</span><span class="p">,</span> <span class="n">until</span><span class="p">,</span> <span class="n">step</span><span class="p">):</span>
<span class="c1"># type: (datetime, datetime, timedelta) -> Generator[datetime, None, None]</span>
<span class="k">while</span> <span class="n">since</span> <span class="o"><</span> <span class="n">until</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">since</span>
<span class="n">since</span> <span class="o">+=</span> <span class="n">step</span>
</code></pre></div>
<p>So it's better to use workflow more o less similar to TTD - check code firstly -> make cheks green -> add new type hints.</p>
<h2>Be ready to ignore type checks for some parts of a code</h2>
<p>Unfortunately, there are complete type hints not for all libraries. Despite a huge amount of work made by a community, not all of them are accurate.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">email.mime.multipart</span> <span class="kn">import</span> <span class="n">MIMEMultipart</span>
<span class="kn">from</span> <span class="nn">email.mime.text</span> <span class="kn">import</span> <span class="n">MIMEText</span>
<span class="k">def</span> <span class="nf">compose_email</span><span class="p">():</span>
<span class="c1"># type: () -> MIMEMultipart</span>
<span class="n">msg</span> <span class="o">=</span> <span class="n">MIMEMultipart</span><span class="p">()</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'Subject'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'Subject'</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'From'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'test@domain.com'</span>
<span class="n">msg</span><span class="p">[</span><span class="s1">'To'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'test@domain.com'</span>
<span class="n">msg</span><span class="o">.</span><span class="n">attach</span><span class="p">(</span><span class="n">MIMEText</span><span class="p">(</span><span class="s1">'Body'</span><span class="p">))</span>
<span class="k">return</span> <span class="n">msg</span>
</code></pre></div>
<p>If you check this code with mypy с with the parameter <code>--py2</code>, it returns some errors:</p>
<div class="highlight"><pre><span></span><code><span class="n">example06</span><span class="p">.</span><span class="nl">py:</span><span class="mh">8</span><span class="o">:</span><span class="w"> </span><span class="nl">error:</span><span class="w"> </span><span class="n">Unsupported</span><span class="w"> </span><span class="n">target</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">indexed</span><span class="w"> </span><span class="n">assignment</span><span class="w"></span>
<span class="n">example06</span><span class="p">.</span><span class="nl">py:</span><span class="mh">9</span><span class="o">:</span><span class="w"> </span><span class="nl">error:</span><span class="w"> </span><span class="n">Unsupported</span><span class="w"> </span><span class="n">target</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">indexed</span><span class="w"> </span><span class="n">assignment</span><span class="w"></span>
<span class="n">example06</span><span class="p">.</span><span class="nl">py:</span><span class="mh">10</span><span class="o">:</span><span class="w"> </span><span class="nl">error:</span><span class="w"> </span><span class="n">Unsupported</span><span class="w"> </span><span class="n">target</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">indexed</span><span class="w"> </span><span class="n">assignment</span><span class="w"></span>
<span class="n">example06</span><span class="p">.</span><span class="nl">py:</span><span class="mh">11</span><span class="o">:</span><span class="w"> </span><span class="nl">error:</span><span class="w"> </span><span class="s">"MIMEMultipart"</span><span class="w"> </span><span class="n">has</span><span class="w"> </span><span class="n">no</span><span class="w"> </span><span class="n">attribute</span><span class="w"> </span><span class="s">"attach"</span><span class="w"></span>
</code></pre></div>
<p>But it doesn't mean that the library is used in a wrong way. No! Stab file for it isn't complete.</p>
<p>If you get such error, you have two options:</p>
<ul>
<li>Fix stab files by yourselves.</li>
<li>Turn off checks for that part of the project.</li>
</ul>
<p>Of course, it's better to make a contribution to the community work and fix type hints for the library. But often there is no time for it. There are deadlines and an infinite list of task. In this case, there are decorators <code>@typing.no_type_check</code> and <code>@typing.no_type_check_decorator</code> in typing library. Using them you can turn off checks for a function, a class or a decorator. For one line there is a type comment: <code># type: ignore</code>.</p>
<h2># type: (...) -> bool</h2>
<p>In Python is OK then predicate returns not exactly a boolean value. Usually, it returns a non-empty object or False. So typical predicate look like this:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Dict</span><span class="p">,</span> <span class="n">Any</span>
<span class="k">def</span> <span class="nf">predicate</span><span class="p">(</span><span class="n">elem</span><span class="p">):</span>
<span class="c1"># type: (Dict[str, Any]) -> bool</span>
<span class="k">return</span> <span class="n">elem</span> <span class="ow">and</span> <span class="s1">'something'</span> <span class="ow">in</span> <span class="n">elem</span>
</code></pre></div>
<p>In a context of using such function <code>bool</code> as a type for the predicate result is a good hint. It fully describes expected predicate behavior. However, the return type is different. In this example it is <code>Union[Dict[str, Any], bool]</code>. So if you add type hinting, you have to cast results to <code>bool</code> explicitly:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Dict</span><span class="p">,</span> <span class="n">Any</span>
<span class="k">def</span> <span class="nf">predicate</span><span class="p">(</span><span class="n">elem</span><span class="p">):</span>
<span class="c1"># type: (Dict[str, Any]) -> bool</span>
<span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="n">elem</span> <span class="ow">and</span> <span class="s1">'something'</span> <span class="ow">in</span> <span class="n">elem</span><span class="p">)</span>
</code></pre></div>
<p>I don't know is it good or bad. On one hand, such code looks strange in Python. On the other hand, it limits function possible usage. Now it's only a predicate, you can't use it in any other context. </p>
<p>In conclusion, I want to remind you that type hinting worth adding only to big projects. For small projects there is the only reason to add type hints - the project is a library.</p>9 Useful Articles about Asyncio2017-10-20T10:00:00+05:002017-10-20T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-10-20:/asyncio-articles.html<p>asyncio is a useful library. However, it is not so easy to understand how to work with it. Documentation isn't enough. We need more examples. We need more explanations. The last couple of weekends I spent experimenting with this library. After that, I add some interesting articles into my Pocket. Here are some of them.</p>
<p>asyncio is a useful library. However, it is not so easy to understand how to work with it. Documentation isn't enough. We need more examples. We need more explanations. The last couple of weekends I spent experimenting with this library. After that, I add some interesting articles into my Pocket. Here are some of them.</p>
<p><em>#1 <a href="http://www.giantflyingsaucer.com/blog/?p=5557">Exploring Python 3’s Asyncio by Example</a></em></p>
<p>A quick intro into asyncio and aiohttp via several short examples. As the author notices, it shows only a tip of the iceberg. Examples in this article are as simple as possible. However, decorators are used in this article. It is a quite old article.</p>
<p><em>#2 <a href="https://compiletoi.net/fast-scraping-in-python-with-asyncio/">Fast scraping in python with asyncio</a></em></p>
<p>This article describes an example implementation of a script that extracts magnet links from web pages. This is also a quite old article. I include in the list because of "the bonus track". The author shows how to limit the number of queries. However, I think it is better to limit the number of queries by aiohttp setting.</p>
<p><em>#3 <a href="http://journalpanic.com/post/wrapping-subprocesses-in-asyncio/">Wrapping Subprocesses in Asyncio</a></em></p>
<p>In this article, a very interesting case is described. The author uses asyncio to handle subprocess stdout. I didn't even think that it is possible to use asyncio to do this. However, the author shows a quite interesting code.</p>
<p><em>#4 <a href="http://www.aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html">A Web Crawler With asyncio Coroutines</a> by A. Jesse Jiryu Davis and Guido van Rossum</em></p>
<p>This is a detailed explanation how to write web crawler using Python and asyncio. In my opinion, there are too much unnecessary explanations and digressions. For example, why do authors explain how generators work? In general, this article is not the best resource about asyncio. I left it the list because of one of the authors.</p>
<p><em>#5 <a href="https://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html">Making 1 million requests with python-aiohttp</a> by Paweł Miech</em></p>
<p>I think the author of this article writes in Go. Why? In Go-world it is OK to write benchmarks for everything. In this article, he tests aiohttp speed limits. By the way, it is 111,111 absolutely useless queries by minute on his hardware. Cool! Nevertheless, it is a very useful article. The author shows some common aiohttp pitfalls and explains how to deal with them.</p>
<p><em>#6 <a href="https://www.pythonsheets.com/notes/python-asyncio.html">Python asyncio cheatsheet</a></em></p>
<p>How to use asyncio by examples. Only code. I doubt you can find a better place to start.</p>
<p><em>#7 <a href="http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/">I don't understand Python's Asyncio</a> by Armin Ronacher</em></p>
<p>Armin complains about the complexity of asyncio. It's true, you have to know a lot of things to use this library 100%. However, my opinion is that the most difficulties will disappear when developers find right approaches how to use it.</p>
<p><em>#8 <a href="https://hackernoon.com/asyncio-for-the-working-python-developer-5c468e6e2e8e">AsyncIO for the Working Python Developer</a> by Yeray Diaz</em></p>
<p>Yet another asyncio tutorial. All examples are written in the new syntax. Each example has a good explanation.</p>
<p><em>#9 <a href="https://medium.com/python-pandemonium/asyncio-coroutine-patterns-beyond-await-a6121486656f">Asyncio Coroutine Patterns: Beyond await</a> by Yeray Diaz</em></p>
<p>This is another article from the same author. This time it is for more experienced developers. It covers several patterns of using asyncio: Recursive coroutines, Fire and forgets, and Periodic coroutines.</p>
<p><em>Bonus:</em> <a href="https://www.datacamp.com/community/tutorials/asyncio-introduction">Asyncio: An Introduction</a> by Mike Driscoll</p>Python project integrity requires extra efforts2017-10-06T19:00:00+05:002017-10-06T19:00:00+05:00akorablevtag:www.alexkorablev.com,2017-10-06:/python-project-integrity.html<p>Python is a very flexible programming language. Its dynamic nature allows programmers to code elegant solutions that almost impossible to make with other more strict languages, for example, Java. However, you have to pay for everything. While Python code base grows, it requires more and more efforts to keep project integrity. Without these extra efforts, the project will fall apart.</p>
<p>Python is a very flexible programming language. Its dynamic nature allows programmers to code elegant solutions that almost impossible to make with other more strict languages, for example, Java. However, you have to pay for everything. While Python code base grows, it requires more and more efforts to keep project integrity. Without these extra efforts, the project will fall apart.</p>
<h2>Problem #1: a code navigation</h2>
<p>The main problem is a code navigation. In this case, I use "code navigation" as a term that includes auto-completion, go-to-declaration and find-usage IDE functions, possibility to use grep through a code base easily. More or less it correlates with the easiness of finding a particular piece of code from any other piece of code in a project.</p>
<p>On early project stages, developers often sacrifice it for a more compact solution. It almost always causes several problems later. One of them is a high cost of debugging.</p>
<p>Traceback in Python isn't the most useful tool. Add some decorators, couple of lambdas, and anything from itertools, and you end up with a bunch of calls that says nothing about how a code really works. In this case, you have to use print statements, grep, and a debugger. All this means that you will lose hours trying to find and localize a problem. </p>
<p>Well, debugging is difficult. It is not a surprising thing. Yes, it is hard. Yes, it takes a lot of time. However, in the end, we'll find what we want. The worst part when you are stuck while writing a code. Just remember the last time you almost recalled how a part of a system worked but you had to find the last tile to complete the puzzle...</p>
<p>And then the chain of calls that took you hours to found it broke in the middle, IDE gave up and showed you "No declaration found" or suggested too many options.</p>
<p>Of course, this happens when you are working on the most complex and tricky part of a project. Of course, 3 years ago it was the most elegant and clean part of the project. But during this time it put on weight, became much more complex, lost flexibility and elegance. And it doesn't matter who decided to implement it this way. You have to deal with it. And it is extremely difficult to do.</p>
<h2>What to do?</h2>
<p>Is there any way how avoid this kind of problems? I have no good answer to it. Perhaps, it wise to follow The Zen of Python: explicit is better than implicit, even if it means to write more code. Do not afraid of adding some Java-style code if it helps to follow The Zen of Python. But basically I think we should follow this two rules:</p>
<h3>Rule #1: You have to have really good reason to break a code navigation</h3>
<p>There are a million ways how to break a code navigation in Python. But you have to have a strong bullet-proof reason to do it. A desire to write a beauty and elegant piece of code isn't enough. Only if there is a high risk to end up with extremely complex code, it is OK to implement some tricky solution.</p>
<p>You should always remember that a code is read more often than written. Keep its readability and integrity is the main priority.</p>
<h3>Rule #2: All places that broke code navigation should be logged carefully</h3>
<p>All tricky solutions should be logged carefully. Make sure that you have all significant details about everything that could help you in future. Log everything that could help you to reconstruct an event sequence that causes any given error in production.</p>
<h3>Type hinting could help, but not much</h3>
<p>You can use type hinting as an additional defensive line. It helps IDE - at least PyCharm - to suggest adequate things. Static analysis allows you to find some tricky problems in your code that you hardly ever find using other methods.</p>
<p>However, type hinting is very hard to use especially if you are working on a big project. It requires obligatory checks for all pull requests. So with regular tests, your CI-server should check types. You can't rely on manual checks. After a week nobody will do them. </p>
<h3>Tools</h3>
<p>Unfortunately, I can't say anything about tools. I doubt there is a tool or metric for a project integrity. mypy is a must-have tool for type checking. But other aspects can be controlled using common sense only.</p>
<p>Tests could help. Not small unit tests. But bigger ones that check interaction of several subsystems. Writing this kind of test often allows me to find places where an integrity and a code navigation fail.</p>
<p>Code review gives good results. To understand code written by someone else you have to jump from one part of it to another. In this condition, all tricky moments show themselves brightly. For me, the main signal about a potential problem is a strong desire to open IDE for code review.</p>
<p>In conclusion, I want to say do not affray of straightforward solutions that easy to read and that doesn't break IDE. They save your and your teammates' nerves and time.</p>Hash function for function in Python2017-09-11T19:00:00+05:002017-09-11T19:00:00+05:00akorablevtag:www.alexkorablev.com,2017-09-11:/function-hash.html<p>A couple of weeks ago one of my colleagues ask if Python function is possible to use as a key in dicts? Yes, it is. So, every function has a hash. But how is it calculated? Based on function name? Based on function byte code? Actually, it calculates by transforming a pointer to a function object. However, it is not so easy to find it in the CPython code.</p>
<p>A couple of weeks ago one of my colleagues ask if Python function is possible to use as a key in dicts? Yes, it is. So, every function has a hash. But how is it calculated? Based on function name? Based on function byte code? Actually, it calculates by transforming a pointer to a function object. However, it is not so easy to find it in the CPython code.</p>
<p>Navigating through C code is an interesting experience and it is not easy at all. There are many macroses that make this process extremely difficult. I am sure you can find out all function that is called if you ask a hash for a function object.</p>
<p>But let's do it a little bit different way. We will go through a couple of calls that are important to understand how CPython works, and then we'll talk about CPython internals a bit. </p>
<p>I'm going to use the master branch of the official <a href="https://github.com/python/cpython">CPython repo</a>. You can clone it or explore it in a browser. The function object is described in <a href="https://github.com/python/cpython/blob/9648088e6ccd6d0cc04f450f55628fd8eda3784c/Objects/funcobject.c">funcobject.c</a>. The most interesting part for us is that:</p>
<div class="highlight"><pre><span></span><code><span class="n">PyTypeObject</span><span class="w"> </span><span class="n">PyFunction_Type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">PyVarObject_HEAD_INIT</span><span class="p">(</span><span class="o">&</span><span class="n">PyType_Type</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="s">"function"</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="k">sizeof</span><span class="p">(</span><span class="n">PyFunctionObject</span><span class="p">),</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="n">destructor</span><span class="p">)</span><span class="n">func_dealloc</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_dealloc */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_print */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_getattr */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_setattr */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_reserved */</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="n">reprfunc</span><span class="p">)</span><span class="n">func_repr</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_repr */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_as_number */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_as_sequence */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_as_mapping */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_hash */</span><span class="w"></span>
<span class="w"> </span><span class="n">function_call</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_call */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_str */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_getattro */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_setattro */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_as_buffer */</span><span class="w"></span>
<span class="w"> </span><span class="n">Py_TPFLAGS_DEFAULT</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Py_TPFLAGS_HAVE_GC</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_flags */</span><span class="w"></span>
<span class="w"> </span><span class="n">func_new__doc__</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_doc */</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="n">traverseproc</span><span class="p">)</span><span class="n">func_traverse</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_traverse */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_clear */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_richcompare */</span><span class="w"></span>
<span class="w"> </span><span class="n">offsetof</span><span class="p">(</span><span class="n">PyFunctionObject</span><span class="p">,</span><span class="w"> </span><span class="n">func_weakreflist</span><span class="p">),</span><span class="w"> </span><span class="cm">/* tp_weaklistoffset */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_iter */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_iternext */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_methods */</span><span class="w"></span>
<span class="w"> </span><span class="n">func_memberlist</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_members */</span><span class="w"></span>
<span class="w"> </span><span class="n">func_getsetlist</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_getset */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_base */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_dict */</span><span class="w"></span>
<span class="w"> </span><span class="n">func_descr_get</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_descr_get */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_descr_set */</span><span class="w"></span>
<span class="w"> </span><span class="n">offsetof</span><span class="p">(</span><span class="n">PyFunctionObject</span><span class="p">,</span><span class="w"> </span><span class="n">func_dict</span><span class="p">),</span><span class="w"> </span><span class="cm">/* tp_dictoffset */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_init */</span><span class="w"></span>
<span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_alloc */</span><span class="w"></span>
<span class="w"> </span><span class="n">func_new</span><span class="p">,</span><span class="w"> </span><span class="cm">/* tp_new */</span><span class="w"></span>
<span class="p">};</span><span class="w"></span>
</code></pre></div>
<p>This is the function type declaration. Each line market by comment <code>tp_something</code> is a pointer to a C function that does something specific for the particular type. However, in our case 0 is used for tp_hash. It means that some inherited function will be called.</p>
<p>It is not so easy to discover what function is used by default. There is only one place where <code>PyFunction_Type</code> structure is used - as a parameter for the <code>PyObject_GC_New</code> macros call inside <code>PyFunction_NewWithQualName</code>.</p>
<p>Although working with C code may be interesting, we will use a shortcut. I suggest looking through <a href="http://ref.readthedocs.io/en/latest/understanding_python/type_system/PyTypeObject.html">"CPython’s PyTypeObject"</a> to find out how PyTypeObject is organized and works. This document says that by default <code>tp_hash</code> source is <code>PyBaseObject_Type.tp_hash</code> and the default value is <code>_Py_HashPointer</code>.</p>
<p>Now we can easily find this function:</p>
<div class="highlight"><pre><span></span><code><span class="n">Py_hash_t</span><span class="w"></span>
<span class="nf">_Py_HashPointer</span><span class="p">(</span><span class="kt">void</span><span class="w"> </span><span class="o">*</span><span class="n">p</span><span class="p">)</span><span class="w"></span>
<span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">Py_hash_t</span><span class="w"> </span><span class="n">x</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="kt">size_t</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">size_t</span><span class="p">)</span><span class="n">p</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="cm">/* bottom 3 or 4 bits are likely to be 0; rotate y by 4 to avoid</span>
<span class="cm"> excessive hash collisions for dicts and sets */</span><span class="w"></span>
<span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">y</span><span class="w"> </span><span class="o">>></span><span class="w"> </span><span class="mi">4</span><span class="p">)</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">(</span><span class="n">y</span><span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="p">(</span><span class="mi">8</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">SIZEOF_VOID_P</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="mi">4</span><span class="p">));</span><span class="w"></span>
<span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">Py_hash_t</span><span class="p">)</span><span class="n">y</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">x</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">-1</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">-2</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">x</span><span class="p">;</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>As you can see, there are no things like qualified function name is used - just its pointer itself.</p>
<p>It was an interesting code discovery challenge. Of course, I've used a shortcut, but it still required some knowledge about CPython internals. One of the best ways to catch up it is watching the <a href="https://www.youtube.com/playlist?list=PLzV58Zm8FuBL6OAv1Yu6AwXZrnsFbbR0S">CPython Internals videos</a>.</p>Success in Programming Book Review2017-08-13T10:00:00+05:002017-08-13T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-08-13:/success-in-programming.html<p>Personal branding isn't a thing that is discussed much in a programming world. Actually, I know very few guys who have strong personal brands and only two guys who can explain what personal branding is for software developers. One of them is Frederic Harper with his book “<a href="https://www.amazon.com/Success-Programming-Recognition-Influence-Personal-ebook/dp/B00MUUI67C">Success in Programming</a>.”</p>
<p>Personal branding isn't a thing that is discussed much in a programming world. Actually, I know very few guys who have strong personal brands and only two guys who can explain what personal branding is for software developers. One of them is Frederic Harper with his book “<a href="https://www.amazon.com/Success-Programming-Recognition-Influence-Personal-ebook/dp/B00MUUI67C">Success in Programming</a>.”</p>
<h2>Why is personal branding important?</h2>
<p>OK. Have you ever saw a guy who got a job in a big tech company without passing all their interview rounds? I know one. Headhunters from one well-known tech company pleased him to spare them a minute. Not for starting an interview process. No! To discuss what project he wanted to join in the company.</p>
<p>He isn't a genius. He is an experienced and very clever developer. However, there are a lot of more clever and more experienced guys. Why the company tried so hard to hire him? The answer I found is a strong personal brand. That's it.</p>
<p>As I know he has been working on his personal brand for 9 or 10 years already. He has a blog, a podcast, an amazing GitHub account. He invests a lot into networking.</p>
<p>Even me, after a couple of years of blogging and social networking, I have much more strong positions among average back-end developer. I can't say that I have a strong brand. However, I've got first results.</p>
<h2>Success in Programming</h2>
<p>Branding is important. So, is there any easy way to build it? Yes and no. In Frederic's book, you'll find a lot of information how to do it as easy as possible. However, there is one the most important and the most difficult thing, finding your identity. It is crucial to succeeding in a personal brand building. The first half of Frederic's book covers this topic.</p>
<p>Basically, you have to answer two “simple” questions:</p>
<ul>
<li>What do you want to do?</li>
<li>What are you doing? </li>
</ul>
<p>It is all about your goals, your passions, your dreams and your personality. For me, it isn't easy at all to find answers. Fortunately, there are several tips in the book that guide you. For example, I wrote a twitter bio, my elevator speech, in 140 symbols.</p>
<p>My “85% Python, 15% Java, 100% Backend” forms the base of my brand. It is not the final version. I'm working on it. However, using it I can stay focused. By the way, your brand isn't something constant. It doesn't mean that if you promote yourself as a backend developer, it will be impossible to become an Android developer tomorrow.</p>
<p>There are a lot of tough questions in this book. However, when you find answers, all other things will become easier. They will transform from unclear scary things into precise tools. At least it will be possible to understand which of them are useful for you.</p>
<p>Unfortunately, the second half of the book is not so cool as the first one. Maybe, it happens because reading it, you already have answers what to do. Nevertheless, there are some very interesting and helpful topics covered by this part, for example, “learn to say no” paragraph with a flowchart representing the algorithm for it.</p>
<p>Although sometimes there is too much Harper on pages, it is a really helpful book. There are a lot of step-by-step tips for software developers how to build a strong personal brand.</p>Data Structures and Algorithms in Java (2nd Edition) Review2017-06-23T19:00:00+05:002017-06-23T19:00:00+05:00akorablevtag:www.alexkorablev.com,2017-06-23:/lafore-java.html<p>For me, the main advantage of "<a href="https://www.amazon.com/Data-Structures-Algorithms-Java-2nd/dp/0672324539">Data Structures and Algorithms in Java</a>" is the language author uses to describe algorithms. He doesn't use overcomplicated academic language with tons of math. No. Robert uses plain English and tries to give an as simple explanation of an algorithm as possible.</p>
<p>For me, the main advantage of "<a href="https://www.amazon.com/Data-Structures-Algorithms-Java-2nd/dp/0672324539">Data Structures and Algorithms in Java</a>" is the language author uses to describe algorithms. He doesn't use overcomplicated academic language with tons of math. No. Robert uses plain English and tries to give an as simple explanation of an algorithm as possible.</p>
<h2>Why this book?</h2>
<p>This book is for busy students or programmers who decided to revise algorithms before an interview. This book can be scanned very fast: just read a list of ideas at the end of a chapter, look through code examples. If you find something you don't understand, read the chapter.</p>
<p>On the other hand, if you read it from cover to cover doing all exercises, you will become experienced in algorithms. The book provides not only text explanation and code listings. It contains a Java application with a demonstration how different algorithms and data structures work. </p>
<p>As I've mentioned above, another reason to read this book is a language. It's written in plain English with an extremely clean explanation of algorithms. If you get sleepy after two pages from Knuth's "The Art of Computer Programming", try this one. </p>
<h2>Is there something that should be better?</h2>
<p>Algorithms is a very conservative topic. There are few changes since the book was published. However, Java has changed a lot. There are new syntax constructions in it. There are new approaches how to write good Java code. There are new code styles.</p>
<p>Please, get me right. All code examples in the book are still valid Java code. Nonetheless, it is not a modern Java. I don't want to see such code in production. Why is it important to use latest approaches in code examples? Because this kind of books is read mostly by students and junior programmers. They copy this style in their projects.</p>
<p>Maybe, it's better to use pseudocode for code examples like Knuth and Cormen do. Of course, it adds extra complexity for students. In this approach, they should translate pseudocode to a usual programming language. In any case, it increases chances that students learn how to use their programming language in a proper manner.</p>
<h3>Pros:</h3>
<ul>
<li>Very easy language</li>
<li>Good set of algorithms</li>
<li>Each chapter has a list of its ideas</li>
</ul>
<h3>Cons:</h3>
<ul>
<li>Rather old</li>
<li>Weird code examples formatting</li>
<li>Some algorithms are given without code, just explanation</li>
</ul>Emacs hot-keys in non-English keyboard layout2017-06-10T10:00:00+05:002017-06-10T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-06-10:/emacs-got-keys.html<p>You are lucky if you need to use standard English keyboard layout only. I use Russian layout as often as English one. Everywhere hot-keys work OK, except Emacs. Maybe because it has an ASCII-is-enough-for-everybody legacy or other reasons, Emacs uses own weird way to switch layouts (or Input Method in Emacs terms): <code>C-\</code>. Instead of using that "standard" way I found another option. </p>
<p>You are lucky if you need to use standard English keyboard layout only. I use Russian layout as often as English one. Everywhere hot-keys work OK, except Emacs. Maybe because it has an ASCII-is-enough-for-everybody legacy or other reasons, Emacs uses own weird way to switch layouts (or Input Method in Emacs terms): <code>C-\</code>. Instead of using that "standard" way I found another option. </p>
<p>This snippet is taken from the post written by <a href="https://twitter.com/novovladimir">Vladimir</a> "<a href="http://reangdblog.blogspot.com/2015/05/emacs.html">Горячие клавиши Emacs в русской раскладке</a>". </p>
<p>To use it just add this code to your Emacs config file:</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="nb">defun</span><span class="w"> </span><span class="nv">reverse-input-method</span><span class="w"> </span><span class="p">(</span><span class="nv">input-method</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="s">"Build the reverse mapping of single letters from INPUT-METHOD."</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="k">interactive</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nf">list</span><span class="w"> </span><span class="p">(</span><span class="nv">read-input-method-name</span><span class="w"> </span><span class="s">"Use input method (default current): "</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="k">and</span><span class="w"> </span><span class="nv">input-method</span><span class="w"> </span><span class="p">(</span><span class="nf">symbolp</span><span class="w"> </span><span class="nv">input-method</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="k">setq</span><span class="w"> </span><span class="nv">input-method</span><span class="w"> </span><span class="p">(</span><span class="nf">symbol-name</span><span class="w"> </span><span class="nv">input-method</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="k">let</span><span class="w"> </span><span class="p">((</span><span class="nv">current</span><span class="w"> </span><span class="nv">current-input-method</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nv">modifiers</span><span class="w"> </span><span class="o">'</span><span class="p">(</span><span class="no">nil</span><span class="w"> </span><span class="p">(</span><span class="nv">control</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="nv">meta</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="nv">control</span><span class="w"> </span><span class="nv">meta</span><span class="p">))))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">when</span><span class="w"> </span><span class="nv">input-method</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nv">activate-input-method</span><span class="w"> </span><span class="nv">input-method</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">when</span><span class="w"> </span><span class="p">(</span><span class="k">and</span><span class="w"> </span><span class="nv">current-input-method</span><span class="w"> </span><span class="nv">quail-keyboard-layout</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">dolist</span><span class="w"> </span><span class="p">(</span><span class="nv">map</span><span class="w"> </span><span class="p">(</span><span class="nf">cdr</span><span class="w"> </span><span class="p">(</span><span class="nv">quail-map</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="k">let*</span><span class="w"> </span><span class="p">((</span><span class="nv">to</span><span class="w"> </span><span class="p">(</span><span class="nf">car</span><span class="w"> </span><span class="nv">map</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nv">from</span><span class="w"> </span><span class="p">(</span><span class="nv">quail-get-translation</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nv">cadr</span><span class="w"> </span><span class="nv">map</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="nf">char-to-string</span><span class="w"> </span><span class="nv">to</span><span class="p">)</span><span class="w"> </span><span class="mi">1</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">when</span><span class="w"> </span><span class="p">(</span><span class="k">and</span><span class="w"> </span><span class="p">(</span><span class="nf">characterp</span><span class="w"> </span><span class="nv">from</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="nf">characterp</span><span class="w"> </span><span class="nv">to</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">dolist</span><span class="w"> </span><span class="p">(</span><span class="nf">mod</span><span class="w"> </span><span class="nv">modifiers</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nf">define-key</span><span class="w"> </span><span class="nv">local-function-key-map</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nf">vector</span><span class="w"> </span><span class="p">(</span><span class="nf">append</span><span class="w"> </span><span class="nf">mod</span><span class="w"> </span><span class="p">(</span><span class="nf">list</span><span class="w"> </span><span class="nv">from</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nf">vector</span><span class="w"> </span><span class="p">(</span><span class="nf">append</span><span class="w"> </span><span class="nf">mod</span><span class="w"> </span><span class="p">(</span><span class="nf">list</span><span class="w"> </span><span class="nv">to</span><span class="p">)))))))))</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nb">when</span><span class="w"> </span><span class="nv">input-method</span><span class="w"></span>
<span class="w"> </span><span class="p">(</span><span class="nv">activate-input-method</span><span class="w"> </span><span class="nv">current</span><span class="p">))))</span><span class="w"></span>
<span class="p">(</span><span class="nv">reverse-input-method</span><span class="w"> </span><span class="ss">'russian-computer</span><span class="p">)</span><span class="w"></span>
</code></pre></div>
<p>If you are a Spacemacs user, add this code into <code>dotspacemacs-configuration-layers</code> in your <code>.spacemacs</code>.</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="nv">reverse-input-method</span><span class="w"></span>
<span class="w"> </span><span class="nb">:location</span><span class="w"> </span><span class="p">(</span><span class="nv">recipe</span><span class="w"></span>
<span class="w"> </span><span class="nb">:fetcher</span><span class="w"> </span><span class="nv">github</span><span class="w"></span>
<span class="w"> </span><span class="nb">:repo</span><span class="w"> </span><span class="s">"avkorablev/reverse-input-method-layer"</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">)</span><span class="w"></span>
</code></pre></div>
<p>and this code into the body of <code>dotspacemacs/user-config</code> function</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="nv">reverse-input-method</span><span class="w"> </span><span class="ss">'russian-computer</span><span class="p">)</span><span class="w"></span>
</code></pre></div>
<p>It works perfectly. The only issue I know is that recalculation of OrgTable formulas isn't working in Russian layout because it is mapped to <code>C-c-*</code> and <code>*</code> change its location.</p>Property in Child Class2017-05-30T11:00:00+05:002017-05-30T11:00:00+05:00akorablevtag:www.alexkorablev.com,2017-05-30:/property-and-inheritance.html<p>It looks obvious that you can override the property in a child class, and call "super" within it. Sounds OK? Yes, it is. However, when my colleague asked me about this behavior, I was confused. Maybe because I am a bit suspicious, every time I saw a magic thing of any kind in a code I expect a trap. What's why I decided to write a simple example to prove myself that it works as it should.</p>
<p>It looks obvious that you can override the property in a child class, and call "super" within it. Sounds OK? Yes, it is. However, when my colleague asked me about this behavior, I was confused. Maybe because I am a bit suspicious, every time I saw a magic thing of any kind in a code I expect a trap. What's why I decided to write a simple example to prove myself that it works as it should.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">print_function</span><span class="p">,</span> <span class="n">unicode_literals</span><span class="p">,</span> <span class="n">absolute_import</span>
<span class="k">class</span> <span class="nc">Parent</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">prop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Parent property"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">prop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">Child</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">prop</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Child property"</span><span class="p">)</span>
<span class="n">child</span> <span class="o">=</span> <span class="n">Child</span><span class="p">()</span>
<span class="n">child</span><span class="o">.</span> <span class="n">prop</span>
</code></pre></div>
<p>Works as expected.</p>
<div class="highlight"><pre><span></span><code>Parent property
Child property
</code></pre></div>
<p>Actually, property decorator just set up an object of a special type instead of function (it's also an object of another special type) and don't touch any internal things in class. So this behavior is expected and clearly described in <a href="https://docs.python.org/3/library/functions.html#property">the documentation</a>. But, who reads docs?</p>Pyenv on Mac2017-05-19T11:00:00+05:002017-05-19T11:00:00+05:00akorablevtag:www.alexkorablev.com,2017-05-19:/pyenv.html<p>It can be painful to manage several Python versions on one laptop. However, it is a very common situation. Usually, developers have many projects that require different Python versions. There are a couple of variants to deal with it. My favourite is <a href="https://github.com/pyenv/pyenv">pyenv</a> with <a href="https://github.com/pyenv/pyenv-virtualenv">virtualenv</a> plugin. I used to use Homebrew itself to do that, but it wasn't so flexible. </p>
<p>It can be painful to manage several Python versions on one laptop. However, it is a very common situation. Usually, developers have many projects that require different Python versions. There are a couple of variants to deal with it. My favourite is <a href="https://github.com/pyenv/pyenv">pyenv</a> with <a href="https://github.com/pyenv/pyenv-virtualenv">virtualenv</a> plugin. I used to use Homebrew itself to do that, but it wasn't so flexible. </p>
<h2>Installation</h2>
<div class="highlight"><pre><span></span><code>$ brew update
$ brew install pyenv
$ brew install pyenv-virtualenv
</code></pre></div>
<div class="highlight"><pre><span></span><code>$ <span class="nb">echo</span> <span class="s1">'eval "$(pyenv init -)"'</span> >> ~/.bash_profile
$ <span class="nb">echo</span> <span class="s1">'eval "$(pyenv virtualenv-init -)"'</span> >> ~/.bash_profile
</code></pre></div>
<h2>Basic usage</h2>
<p>I strongly recommend making a separate virtual environment for each project. Sometimes it is necessary to make several environments for one project. For example, if you want to use mypy for a Python 2 project.</p>
<p>First of all get the list with available versions.</p>
<div class="highlight"><pre><span></span><code>$ pyenv install --list
</code></pre></div>
<p>There are a lot of variants including PyPy and Stackless Python. If the latest version is not in the list, update Homebrew formula:</p>
<div class="highlight"><pre><span></span><code>$ brew upgrade pyenv
</code></pre></div>
<p>Usually, new Python versions appear in the list in a day or two.</p>
<p>There are 3 steps to prepare virtualenv for your project:</p>
<ol>
<li>Install new Python version if needed</li>
<li>Make new virtualenv</li>
<li>Activate environment</li>
</ol>
<div class="highlight"><pre><span></span><code>$ pyenv install <span class="m">3</span>.6.0
$ pyenv virtualenv <span class="m">3</span>.6.0 my-project-venv
$ <span class="nb">cd</span> /to/my/project/folder
$ pyenv <span class="nb">local</span> my-project-venv
</code></pre></div>
<p>That's it.</p>
<p>In case of using PyCharm, you should create a new Python SDK with home path equal to <code>~/.pyenv/versions/my-project-venv</code></p>Big Data. Educational Resources2017-04-28T11:00:00+05:002017-04-28T11:00:00+05:00akorablevtag:www.alexkorablev.com,2017-04-28:/big-data.html<p>Big Data is a kind of buzzword nowadays. However, it looks that having skills in it can make a huge influence on a programming career. At least, it's true for me. That's why this list appears. It's a unsorted and unfiltered bunch of resources covered "Big Data" topic.</p>
<p>Big Data is a kind of buzzword nowadays. However, it looks that having skills in it can make a huge influence on a programming career. At least, it's true for me. That's why this list appears. It's a unsorted and unfiltered bunch of resources covered "Big Data" topic.</p>
<h2>Computer Science Fundamentals and Programming</h2>
<p><a href="https://www.hackerrank.com/domains/algorithms">Algorithm challanges on HackerRank</a></p>
<p><a href="https://www.hackerrank.com/domains/data-structures">Datastructure challanges on HackerRank</a></p>
<p><a href="https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press/dp/0262033844">Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein</a></p>
<p><a href="https://www.coursera.org/specializations/scala">Functional Programming in Scala Specialization</a></p>
<p><a href="https://www.coursera.org/specializations/cloud-computing">Cloud Computing Specialization</a></p>
<p><a href="https://www.coursera.org/specializations/algorithms">Algorithms Specialization</a></p>
<h2>Applied Math</h2>
<p><a href="https://www.youtube.com/playlist?list=PLQ3khvAsNhargDx0dG1cQXOrA2u3JsFKc">Probability and Statistics</a></p>
<p><a href="https://www.coursera.org/learn/datasciencemathskills">Data Science Math Skills</a></p>
<p><a href="https://www.coursera.org/learn/bayesian-statistics">Bayesian Statistics: From Concept to Data Analysis</a></p>
<p><a href="https://www.coursera.org/learn/linear-models">Advanced Linear Models for Data Science 1: Least Squares</a></p>
<p><a href="https://www.coursera.org/learn/linear-models-2">Advanced Linear Models for Data Science 2: Statistical Linear Models</a></p>
<h2>Data Science</h2>
<p><a href="https://www.coursera.org/specializations/jhu-data-science">Data Science Specialization</a></p>
<p><a href="https://www.coursera.org/specializations/recommender-systems">Recommender Systems Specialization</a></p>
<p><a href="https://www.udacity.com/course/intro-to-data-analysis--ud170">Intro to Data Analysis</a></p>
<h2>Machine Learning</h2>
<p><a href="https://www.youtube.com/playlist?list=PLA89DCFA6ADACE599">Machine Learning Lectures</a></p>
<p><a href="https://www.udacity.com/course/deep-learning--ud730">Deep Learning</a></p>
<p><a href="https://www.coursera.org/specializations/machine-learning">Machine Learning Specializatio</a></p>
<p><a href="https://www.coursera.org/learn/machine-learning">Machine Learning</a></p>
<p><strong>IN RUSSIAN</strong> <a href="https://www.coursera.org/specializations/machine-learning-data-analysis">Машинное обучение и анализ данных</a></p>
<h2>Frameworks</h2>
<p><a href="https://www.edx.org/xseries/data-science-engineering-apacher-sparktm">Data Science and Engineering with Apache Spark</a></p>
<p><a href="https://www.futurelearn.com/courses/big-data-r-hadoop">Managing Big Data with R and Hadoop</a></p>
<p><a href="https://bigdatauniversity.com/courses/deep-learning-tensorflow/">Deep Learning with TensorFlow</a></p>
<p><a href="https://www.udacity.com/course/intro-to-hadoop-and-mapreduce--ud617">Intro to Hadoop and MapReduce</a></p>
<p><a href="https://www.udacity.com/course/deploying-a-hadoop-cluster--ud1000">Deploying a Hadoop Cluster</a></p>
<p><a href="http://shop.oreilly.com/product/0636920071440.do">Data Science with Spark</a></p>
<p><a href="http://shop.oreilly.com/product/9781785885655.do">Spark for Data Science</a></p>
<p><a href="http://shop.oreilly.com/product/0636920028512.do">Learning Spark</a></p>
<h2>Other</h2>
<p><a href="https://www.coursera.org/specializations/big-data">Big Data Specialization</a></p>
<p><a href="https://www.amazon.com/Scala-Data-Science-Pascal-Bugnion/dp/1785281372">Scala for Data Science</a></p>
<p><a href="https://www.amazon.co.uk/Scala-Machine-Learning-Patrick-Nicolas/dp/1783558741">Scala for Machine Learning</a> </p>
<p><a href="http://shop.oreilly.com/product/0636920034919.do">Python Data Science Handbook</a></p>
<p><a href="http://shop.oreilly.com/product/0636920030515.do">Introduction to Machine Learning with Python</a></p>
<p><a href="http://shop.oreilly.com/product/9780596516499.do">Natural Language Processing with Python</a> </p>
<p><a href="https://www.kaggle.com">Kaggle</a></p>
<h2>Resources</h2>
<p><a href="http://www.datasciencecentral.com/profiles/blogs/20-articles-about-core-data-science">What is Data Science? 24 Fundamental Articles Answering This Question</a></p>
<p><a href="https://www.quora.com/What-skills-are-needed-for-machine-learning-jobs">Quora: What skills are needed for machine learning jobs?</a></p>
<p><a href="https://www.quora.com/How-do-I-learn-big-data-technologies-1">Quora: How do I learn big data technologies?</a></p>
<p><a href="https://www.quora.com/Is-there-any-website-where-we-can-learn-Data-Science-for-free">Quora: Is there any website where we can learn Data Science for free?</a></p>
<p><a href="https://www.quora.com/Where-should-I-start-from-for-learning-big-data-development">Quora: Where should I start from for learning big data development?</a></p>
<p><a href="http://blog.udacity.com/2016/04/5-skills-you-need-to-become-a-machine-learning-engineer.html">5 Skills You Need to Become a Machine Learning Engineer</a></p>
<p><a href="http://bigdata-madesimple.com/7-key-skills-required-for-machine-learning-jobs/">7 key skills required for Machine Learning jobs</a></p>Soft Skills Book Review2017-04-15T13:00:00+05:002017-04-15T13:00:00+05:00akorablevtag:www.alexkorablev.com,2017-04-15:/soft-skills.html<p>When I started reading <a href="https://www.amazon.com/Soft-Skills-software-developers-manual/dp/1617292397">"Soft Skills" by John Sonmez</a>, I didn't know what to expect from it. On the one hand, as software engineers, we think that writing beauty code is much more important than knowledge of how to promote and market ourselves. And it is true. Although, It is extremely hard to find an interesting project without marketing and self-promotion. I doubted that someone could explain this thing to developers. I can't say that John does it ideally, but he does it quite well.</p>
<p>When I started reading <a href="https://www.amazon.com/Soft-Skills-software-developers-manual/dp/1617292397">"Soft Skills" by John Sonmez</a>, I didn't know what to expect from it. On the one hand, as software engineers, we think that writing beauty code is much more important than knowledge of how to promote and market ourselves. And it is true. Although, It is extremely hard to find an interesting project without marketing and self-promotion. I doubted that someone could explain this thing to developers. I can't say that John does it ideally, but he does it quite well.</p>
<p>First of all, the author of the book, John Sonmez, was a software engineer. Now he is a coach, a trainer, and a real estate investor, I suppose. He says that he works as developer sometimes. Anyway, he is the guy who knows the industry inside. He has a very interesting story of his career. Although he wasn't CEO of Microsoft or Google, his advice deserves to be checked. Maybe, it's better that he wasn't a CEO in any huge corporation. That makes him more like to an average software engineer.</p>
<p>OK. He knows something about software development. Does he know something about personal branding and marketing? I can't answer this question. Before I read the book, I'd read some John's blog posts. Basically, he doesn't write anything extra-ordinal. Actually, after almost any marketing tip you can exclaim "It's obvious!", "Everybody knows it!" Instead of giving weird and complicated advice, John adds an example after each simple, plain advice he gave to prove it.</p>
<p>Not all parts of the book have the same quality. The better John knows a topic of a chapter, the cleaner explanation, better tips and examples he gives. Sections "Career", "Marketing yourself", "Learning", "Productivity" are great. </p>
<p>Starting from "Financial" you'll find that some chapters are not so good. I mean, that the chapter named "How to negotiate your salary" or one about real estate are brilliant. In the same time, the book contains chapter "Options: Where all the fun is" is the most controversial and dangerous in the book. I want to say that options aren't so fun. Basically, they are extremely complex and risky for amateurs. Of course, you can use them successfully, but you must be absolutely sure what you do.</p>
<p>Last to sections are about health and mental state. "Fitness" section is good. John is a well-built man. And it's wise to read carefully and use in your life what he says about setting fitness goals, workouts, running, and diet.</p>
<p>In conclusion, I want to answer two questions. The first one is "Is it worth reading?" Yes, it is. There are many good tips inside this book. Just chose one that looks more useful and apply it. If it works, choose another one and repeat. The second question is "Is this the best book you've ever read?" Of course, not. However, I found only two books about personal marketing for software engineers. That's why we actually have no choice and we should read it.</p>Is this MOOC good for me?2017-04-05T17:00:00+05:002017-04-05T17:00:00+05:00akorablevtag:www.alexkorablev.com,2017-04-05:/mooc.html<p>There are a lot of massive open online courses (MOOCs) today. Some of them are free, other paid. Such courses were all hype couple years ago. Now when we all come down to earth, there are some questions about that courses: how to add them to my education plan and how to choose the best one.</p>
<p>There are a lot of massive open online courses (MOOCs) today. Some of them are free, other paid. Such courses were all hype couple years ago. Now when we all come down to earth, there are some questions about that courses: how to add them to my education plan and how to choose the best one.</p>
<blockquote>
<p>All animals are equal, but some animals are more equal than others definition. </p>
<p>Animal Farm, by George Orwell</p>
</blockquote>
<p>Almost all MOOCs have the same structure:</p>
<ol>
<li>They have educational materials: video or text lectures, code examples, and so on</li>
<li>They have exercises: tests, code assignments</li>
<li>They provide a connection between students and teachers</li>
</ol>
<p>From this point of view, all courses are equal. But a lack of any component could make the course less valuable or even useless. If lectures aren't relevant, event brilliant exercises will not make course good. Vice versa, a bunch of good lectures without exercises is just a bunch of good lectures. They couldn't give you any relevant experience. They just give you dead knowledge.</p>
<p>I say about the case then the quality of materials are OK. But sometimes happens that MukundKumar Mishra <a href="https://www.quora.com/What-can-be-some-good-pet-projects-to-learn-Apache-Spark/answer/MukundKumar-Mishra">has written about</a>:</p>
<blockquote>
<p>As soon as a new thing comes into industry. Many company start claiming real project based training. The problem with beginner is those are not aware of what should be the right course and market is flooded with courses which are based on something industry do not need. Till beginners understand those are cheated the fast growing IT industry moves on and the delay causes irreparable loss.</p>
</blockquote>
<p>To avoid this trap I have developed several simple rules. I follow them every time I decide to join a course. </p>
<h2>Choose the right platform</h2>
<p>I prefer well-known platforms: Udemy, Pluralsight, Coursera and Stepic. The last two are my personal favorites. Stepic contains courses mostly in Russian. But both platforms have similar features: video lectures, tests, and auto-checked code assignments.</p>
<p>For most programming courses auto-checked assignment is crucial. It gives you instant feedback. Although it's impossible or extremely hard to develop good automatic tests for some topics, I rather choose a course with bad tests than without any tests. Of course, I'll prefer code review made by someone from a course team, but such courses are rare. In some cases, tests could replace code assignments, but such courses are significant less valuable for me.</p>
<h2>Choose the right author</h2>
<p>It's very easy to do if we tell about Coursera. Only colleges and universities can make courses for that platform. Professors adopt or make new courses for Coursera. This platform has filtered out trash courses. You could find there a course that isn't OK. But it very difficult to find there a course that absolutely irrelevant.</p>
<p>Other platforms without so tight selection demand more accurate course selection. I usually check authors info: their experience, blogs, linked profiles and so on. I check everything I found. Basically, I try to understand whether the author knows the ropes.</p>
<p>Don't forget to find out other courses from the author. I believe in specialization. If an author has courses in every topic you could imagine, I'll not buy any of these courses. It's impossible to make a good course in Java Spring Framework and a good one in derivative trading. I even hesitate about a possibility for one guy to make good Frontend and Backend courses.</p>
<h2>Choose the right course</h2>
<p>Finding information about a course may be tricky. Sometimes, the only information you have is user reviews and ratings on a platform. I usually read several 5-star reviews, several 1-star reviews. It gives me some information about what is good and bad in the course. Then I read a couple of 3-star or 4-star reviews. Usually, they are less emotional, more balanced, and more detailed. </p>
<p>However, I'm a bit paranoid. Event working with popular MOOC platforms I'm not sure about the quality of user reviews. For me, it is important to find course reviews on any independent site. Quora is one of the best places for it.</p>
<p>In conclusion, I should say that these rules are flexible. If a course covers a very interesting narrow topic, it is OK for me if nobody's written a review on Quora. Although I'll check author profile anyway.</p>Modern Operating Systems 4th edition Review2017-03-19T10:00:00+05:002017-03-19T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-03-19:/tanenbaum.html<p>If you want to know how modern operating systems work, you should read <a href="https://www.amazon.com/Modern-Operating-Systems-Andrew-Tanenbaum/dp/013359162X">Modern Operating Systems</a> written by Tanenbaum and Bos. It is a well-written book full of actual information about its topic.</p>
<p>If you want to know how modern operating systems work, you should read <a href="https://www.amazon.com/Modern-Operating-Systems-Andrew-Tanenbaum/dp/013359162X">Modern Operating Systems</a> written by Tanenbaum and Bos. It is a well-written book full of actual information about its topic.</p>
<p>Each chapter of the book deserves to be a separate book. With a few exceptions, all chapters have the same structure: intro with a quick view to chapter's topic, main part, short list of newest researches connected with the chapter. Every chapter ends with a list of exercises and questions to discuss.</p>
<p>It is a book for students. Authors tell us about topics covered by the book enough to understand main ideas, but not so much deep to get bored. No doubt, it could be used as textbook for university course or as a self-study book. </p>
<p>But not only students benefit from this book. IT professionals will find this book interesting and useful. It gives clear explanation how real operating systems work. It shows common things and differences between Linux, Windows, and Android. It's very sad but there aren't chapters about MacOS and iOS.</p>
<p>Of course, it is not a book for operating system developers. But if you are just interested in its topic, it is worth to read.</p>Enums in Python2017-02-17T10:00:00+05:002017-02-17T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-02-17:/enums.html<p>Enums are less used feature in Python. We as programmers prefer to use weird dicts or lists where enum should be used. Why? Because it is rather new feature and you have to install back port library if you use Python 2.7. However, it's better to use them in many cases.</p>
<p>Enums are less used feature in Python. We as programmers prefer to use weird dicts or lists where enum should be used. Why? Because it is rather new feature and you have to install back port library if you use Python 2.7. However, it's better to use them in many cases.</p>
<p>It is easy to find in any Python code several checks against string literals. We use such things to detect post types, client roles and so on. This code usually tends to change in time. We change set of used literals. That makes such code hard to test and maintain.</p>
<p>For example, we have such code. </p>
<div class="highlight"><pre><span></span><code><span class="k">if</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span> <span class="o">==</span> <span class="s1">'article'</span><span class="p">:</span>
<span class="n">do_something_if_article</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span> <span class="o">==</span> <span class="s1">'review'</span><span class="p">:</span>
<span class="n">do_something_if_review</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">do_something</span><span class="p">()</span>
</code></pre></div>
<p>Is it OK? Not really. What if we decided to change type name from "review" to "reviews". In this case, instead of executing <code>do_something_if_review</code> function we'll execute <code>do_something</code>. Even worse this code isn't easy to refactor. Name of the type could be used in many situations and variants. Could we make this code better? Sure!</p>
<div class="highlight"><pre><span></span><code><span class="n">ARTICLE_TYPE</span> <span class="o">=</span> <span class="s1">'article'</span>
<span class="n">REVIEW_TYPE</span> <span class="o">=</span> <span class="s1">'review'</span>
<span class="k">if</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span> <span class="o">==</span> <span class="n">ARTICLE_TYPE</span><span class="p">:</span>
<span class="n">do_something_if_article</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span> <span class="o">==</span> <span class="n">REVIEW_TYPE</span><span class="p">:</span>
<span class="n">do_something_if_review</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">do_something</span><span class="p">()</span>
</code></pre></div>
<p>Now we use constants. It's much better. Know we have only one place where we have to change. What if we have to add new type? We should add another elif-branch. Not so cool, is it?</p>
<p>This code could be improved using enums:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">PostType</span><span class="p">(</span><span class="n">Enum</span><span class="p">):</span>
<span class="n">ARTICLE</span> <span class="o">=</span> <span class="n">do_something_if_article</span>
<span class="n">REVIEW</span> <span class="o">=</span> <span class="n">do_something_if_review</span>
<span class="n">OTHER</span> <span class="o">=</span> <span class="n">do_something</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">of</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">type_name</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">type_name</span><span class="o">.</span><span class="n">upper</span><span class="p">(),</span> <span class="bp">cls</span><span class="o">.</span><span class="n">OTHER</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">do_stuff</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">value</span><span class="p">()</span>
<span class="n">PostType</span><span class="o">.</span><span class="n">of</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="p">)</span><span class="o">.</span><span class="n">do_stuff</span><span class="p">()</span>
</code></pre></div>
<p>It looks nice and this code is very flexible. I'm sure you have many places in your projects that you can improve using enums. Let's do it!</p>
<p>P.S. <code>pip install enum34</code> for Python 2.7</p>Why I Use Plain Text Everywhere2017-01-31T10:00:00+05:002017-01-31T10:00:00+05:00akorablevtag:www.alexkorablev.com,2017-01-31:/plain-text.html<p>A week ago I tried to migrate the blog to Wordpress again. I decided to give it another chance because I had plans to add more images to posts. It's quite difficult to do it using <a href="http://docs.getpelican.com/en/stable/">Pelican</a>. Wordpress has a great tool for image managing. I thought that its WYSIWYG editor was good too. I installed it and I even converted a couple of posts to it. But then I decided to stop. It took me 20 minutes to format properly that posts.</p>
<p>A week ago I tried to migrate the blog to Wordpress again. I decided to give it another chance because I had plans to add more images to posts. It's quite difficult to do it using <a href="http://docs.getpelican.com/en/stable/">Pelican</a>. Wordpress has a great tool for image managing. I thought that its WYSIWYG editor was good too. I installed it and I even converted a couple of posts to it. But then I decided to stop. It took me 20 minutes to format properly that posts.</p>
<p>Actually, it's much easier for me to use plain text for blogging. Why? It takes only a couple of minutes to format each post. Image placing adds about 5-10 minutes to it. And I could edit drafts everywhere and I could save it in a git repo like I do with source code. But the most important reason is that I can write much faster.</p>
<p>I don't know why, but all WYSIWYG editors slow me down and I prefer plain text to any kind of "rich" format. Maybe, I've spent many years coding, or I waste too much time formatting text in WYSIWYG. However, I use plain text everywhere where it's possible.</p>
<p>For example, I switched from Evernote to Simplenote for note taking and writing blog post drafts. I hardly ever keep the images. Evernote is now too big and too slow. OneNote is the same. Google Keep is fast and looks like a good choice, but it haven't got any API. That's why I choose Simplenote. It has API, I want to have possibility to get all my notes without any problems, and it is fast.</p>
<p>The second field where I use plain text is blogging. As I say before I use plain text for writing blog post drafts. And instead of using Wordpress for publishing it, I use Pelican for that. I've tried both, but it's very hard to make post look good, especially post with code example. Markdown and Pelican make the publishing process easier for me.</p>
<p>And of course I use <a href="http://orgmode.org">Orgmode</a> and Taskpaper as planning tools. Both formats are extremely flexible. Because I use Emacs, I could use all Orgmode features. But it remains just plain text file anyway. It could be edited in every text editor on any existing platform.</p>
<p>Plain text isn't for everyone. Almost every task needs some workaround if you use plain text. You have to be a geek a little bit to handle it.</p>Effective Java 2nd edition Book Review2016-12-30T10:00:00+05:002016-12-30T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-12-30:/effective-java.html<p>If you ask what Java book you must read, everyone says <a href="https://www.amazon.com/Effective-Java-2nd-Joshua-Bloch/dp/0321356683">Effective Java</a>. This book is in all lists of recommended Java books except the lists witch author sells own books. There are several reasons why this quite old book is so popular.</p>
<p>If you ask what Java book you must read, everyone says <a href="https://www.amazon.com/Effective-Java-2nd-Joshua-Bloch/dp/0321356683">Effective Java</a>. This book is in all lists of recommended Java books except the lists witch author sells own books. There are several reasons why this quite old book is so popular.</p>
<p>Firstly, it teaches you how to write good Java code. It means not only faster code, but readable and maintainable code. It describes common Java patterns that make your code easy to understand to other engineers. This book doesn't tell you how to hack JMV, it tells you how to use it effectively.</p>
<p>Secondly, although Mr. Block don't tell you about Stream API or Lambda expressions (we all have to wait for the third edition), he covers other complex topics such as Generics, Concurrency, or Serialization. Almost everything from this book you can apply in your everyday work.</p>
<p>On the other side, this book is about Java 6. There a lot of things that are new in Java 7 and 8. For example, Stream API is needed to be described in a book like this one. I hope there will be 3rd edition soon.</p>
<p>In conclusion, Effective Java isn't huge book. It's just about 350 pages. It's worth a week to read it.</p>How to set up HTTPS for S3 static site2016-12-12T17:00:00+05:002016-12-12T17:00:00+05:00Alex Korablevtag:www.alexkorablev.com,2016-12-12:/s3-https-blog.html<p>Google is using HTTPS as a ranking signal since <a href="https://webmasters.googleblog.com/2014/08/https-as-ranking-signal.html">August 2016</a>. Amazon lets their users free certificates. This two factors forces me to find out how to set up HTTPS for my S3 blogs. The solution is to place an S3 bucket behind CloudFront.</p>
<p>Google is using HTTPS as a ranking signal since <a href="https://webmasters.googleblog.com/2014/08/https-as-ranking-signal.html">August 2016</a>. Amazon lets their users free certificates. This two factors forces me to find out how to set up HTTPS for my S3 blogs. The solution is to place an S3 bucket behind CloudFront.</p>
<h2>Before start</h2>
<p>CloudFront is a caching service. You should be patient while working with it. If you make a mistake, it takes about 20-25 minutes before changed configuration is applied.</p>
<p>My experience is based on setting up HTTPS for this blog. I run it with the pelican static generator. It should be a similar process for Jekyll and other generators.</p>
<h2>Step #1. Request certificate with ACM</h2>
<p>Go to <a href="https://console.aws.amazon.com/acm/home?region=us-east-1#/">ACM console</a> and request a certificate. I hope you have an access to domain owner's email address. Amazon sends an email with a verification link using this address.</p>
<p><strong>Note!</strong> If you are using domain.com with out www. as I do, request certificate for domain.com without "www." or "*." before it.</p>
<h2>Step #2. Create CloudFront distribution for Web delivery</h2>
<p>Go to CloudFront console and select "Create new Distribution".</p>
<p>First, it oferred you to select a delivery method. Choose Web.</p>
<p><img alt="Delivery method" src="https://www.alexkorablev.com/images/cloud_front_1.png"></p>
<p>On the second screen, select S3 bucket with website content in the Origin Domain Name. And Possible you should set "Restrict Bucket Access" to Yes. But I don't check it.</p>
<p><img alt="Origin domain name" src="https://www.alexkorablev.com/images/cloud_front_2.png"></p>
<p>Scroll down to "Default Cache Behavior Settings". Change "Viewer Protocol Policy" to "Redirect HTTP to HTTPS". </p>
<p><img alt="Protocol policy" src="https://www.alexkorablev.com/images/cloud_front_3.png"></p>
<p>In the "Distribution Settings" enter your domain name in Alternate Domain Names and choose Custom SSL Certificate (If there are no available certificates, go to Step #1).</p>
<p><img alt="Distribution settings" src="https://www.alexkorablev.com/images/cloud_front_4.png"></p>
<p>And the last thing you should do is to add "index.html" into Default Root Object. </p>
<p><img alt="Default root object" src="https://www.alexkorablev.com/images/cloud_front_5.png"></p>
<p>Now enable distribution. Make a cup of coffee. It takes about 20-25 minutes before distribution becomes ready to use.</p>
<h2>Step #3. Change Route 53 zone.</h2>
<p>After the distribution was deployed, change Route 53 settings. Set appropriate A record target for CloudFront distribution.</p>
<p>That's all. Your blog is HTTPS now.</p>
<h2>TODO</h2>
<p>There are a couple of things to be solved:</p>
<ol>
<li>I don't know how to set up redirects from www.domain.com to domain.com.</li>
<li>CloudFront is a caching proxy. You have to invalidate cache after content update. I haven't found a good way to do it yet using pelican.</li>
</ol>Java-like Optional in Python2016-11-23T17:00:00+05:002016-11-23T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-11-23:/optional-in-python.html<p>Sometimes using None makes a mess of a code. For example, if you want to separate some "empty" value and "no data" value for integer input where 0 isn't "empty" value, it's very hard to do with None only. Possible solution is to use something like Optional in Java.</p>
<p>Sometimes using None makes a mess of a code. For example, if you want to separate some "empty" value and "no data" value for integer input where 0 isn't "empty" value, it's very hard to do with None only. Possible solution is to use something like Optional in Java.</p>
<p>Of course, using Optional isn't the pythonic way. In some cases using exception as supposed in <a class="reference external" href="http://stackoverflow.com/questions/22992433/is-there-a-python-equivalent-for-scalas-option-or-either">this StackOverflow discussion</a> is enough.</p>
<p>It doesn't work in my case. I decided to implement Optional. At the end I've got clean and nice piece of code. After short discussion with my colleagues, we decided to use Haskell names for it (Maybe instead Optional). There is my simple version:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">abc</span> <span class="kn">import</span> <span class="n">ABCMeta</span><span class="p">,</span> <span class="n">abstractmethod</span>
<span class="k">class</span> <span class="nc">Maybe</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="n">__metaclass__</span> <span class="o">=</span> <span class="n">ABCMeta</span>
<span class="nd">@abstractmethod</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">Just</span><span class="p">(</span><span class="n">Maybe</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">def</span> <span class="nf">__nonzero__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="kc">True</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">value</span>
<span class="k">class</span> <span class="nc">Nothing</span><span class="p">(</span><span class="n">Maybe</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__nonzero__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">return</span> <span class="n">default</span>
</pre></div>
<p>After integrating it to existing code, I can say that this solution works perfectly. Maybe you should add repr-functions. But even without them it's very convenient while debug.</p>
<p>If you want something more complex, try <a class="reference external" href="https://github.com/billpmurphy/hask">hask</a>. Note! There is too much Haskell in their Python.</p>
<p>Some more approaches to deal with the same problem you can find in <a class="reference external" href="https://glyph.twistedmatrix.com/2015/09/python-option-types.html">Python Option Types article.</a></p>
Backtracing without Recusion in Python2016-11-03T17:00:00+05:002016-11-03T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-11-03:/python-backtracking.html<p>As you know there is no tail recursion optimization in Python. Moreover it has a very low recursion limit. It can be a problem if you are trying to solve the problem using the backtracking algorithm. Recursion limit is big enough to solve sudoku, but an exception will raise if the problem has more possible solutions.</p>
<p>As you know there is no tail recursion optimization in Python. Moreover it has a very low recursion limit. It can be a problem if you are trying to solve the problem using the backtracking algorithm. Recursion limit is big enough to solve sudoku, but an exception will raise if the problem has more possible solutions.</p>
<p>However implementation of backtracking in Python with recursion is elegant. It's easy to read and understand.</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">backtrack</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">solution</span><span class="p">):</span>
<span class="k">if</span> <span class="n">solution</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">seen_solutions</span><span class="p">:</span>
<span class="k">return</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">reject</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="k">return</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">accept</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">output</span><span class="p">(</span><span class="n">solution</span><span class="p">)</span>
<span class="k">for</span> <span class="n">child_solution</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">child_solutions</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">backtrack</span><span class="p">(</span><span class="n">child_solution</span><span class="p">)</span>
</pre></div>
<p>When you rich recursion limit, you can switch to this template without recursion. It doesn't look so clean as one with recursion, but it's not so ugly than it could be, and it uses the same auxiliary functions.</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">backtrack</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">solution</span><span class="p">):</span>
<span class="n">solution_stack</span> <span class="o">=</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">child_solutions</span><span class="p">(</span><span class="n">solution</span><span class="p">)]</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">current_generator</span> <span class="o">=</span> <span class="n">solution_stack</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="n">solution</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">current_generator</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">while</span> <span class="n">solution</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">reject</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="n">solution</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">current_generator</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">continue</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">accept</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">output</span><span class="p">(</span><span class="n">solution</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">solution_is_complete</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="n">solution_stack</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">child_solutions</span><span class="p">(</span><span class="n">solution</span><span class="p">))</span>
<span class="n">current_generator</span> <span class="o">=</span> <span class="n">solution_stack</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="n">solution</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">current_generator</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="n">solution_stack</span> <span class="o">=</span> <span class="n">solution_stack</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">solution_stack</span><span class="p">:</span>
<span class="k">return</span>
</pre></div>
Functional Programming Principles in Scala Course Review2016-10-24T17:00:00+05:002016-10-24T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-10-24:/progfun1.html<p>This fall I finished <a class="reference external" href="https://www.coursera.org/learn/progfun1">"Functional Programming Principles in Scala"</a> Coursera course. This course is a part of specialization "Functional Programming in Scala." It's 6 weeks long introduction to Scala and functional programming.</p>
<p>This fall I finished <a class="reference external" href="https://www.coursera.org/learn/progfun1">"Functional Programming Principles in Scala"</a> Coursera course. This course is a part of specialization "Functional Programming in Scala." It's 6 weeks long introduction to Scala and functional programming.</p>
<p>Author of this course is Martin Odersky, Scala language creator. He is a great scientist. Nevertheless I felt sleepy watching his lectures. Actually there is nothing wrong with his lectures. I thing it's better to change way to pass the course: first try to solve the problem from programming assignment, then watch the lecture covers that topic.</p>
<p>The structure of the course is straightforward. First week covers basics of Scala and difference between imperative and functional programming. Rest weeks cover topics such: high order functions, traits and classes, types and pattern matching, and of course how to deal with lists and other collections in Scala.</p>
<p>Every week of the course is ended up by programming assignment. Mainly they aren't difficult. However, some assignments require more effort if you want to get the best mark. Note that if all local tests are passed, it does't mean that all tests will be passed on Coursera server.</p>
<p>There are more tests, there than you have in assignment templates. I strongly recommend adding additional tests. It's not easy sometimes, but it increases course value dramatically.</p>
<p>In conclusion, I want to say if after the course you decide that Scala isn't for you, just take the next course in the specialization.</p>
Want to add type annotation to a variable? Do refactoring!2016-10-06T17:00:00+05:002016-10-06T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-10-06:/local-variable-type.html<p>There is the <a class="reference external" href="https://www.python.org/dev/peps/pep-0526/">PEP 526</a> implemented in Python 3.6 that adds optional type annotations for variables. It's replacement for type comments. There is one pitfall: adding type annotations for local variables can hide problems with code.</p>
<p>There is the <a class="reference external" href="https://www.python.org/dev/peps/pep-0526/">PEP 526</a> implemented in Python 3.6 that adds optional type annotations for variables. It's replacement for type comments. There is one pitfall: adding type annotations for local variables can hide problems with code.</p>
<p>Such annotations are made for robots, for automatic code checkers. And if you use it to increase code readability, you are in trouble. It must be your last weapon, not the first one.</p>
<p>If you think that an annotation makes the code better, try to discover that is the problem. In most cases it's one of two possible variants: either variable has complex type or value for this variable is unclear.</p>
<p>In the case with complex type, you can try to extract custom class or use namedtuple or something like this.</p>
<p>In the second case, try to split the function. Or rewrite it more clearly.</p>
<p>For example, I love using "map", "sorted", "filter", and other things from functool. Sometimes I write big constructions that transform a data beyond recognition.</p>
<p>Of course, if I add type annotation for the result variable, it makes code more clear. But I think better to split such constructions up into smaller, cleaner steps, and give that steps good reasonable names.</p>
MongoDB count queries can slow down your API2016-09-23T10:00:00+05:002016-09-23T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-09-23:/mongo-counters.html<p>Sometimes front-end team or other API clients ask to show them total number of objects in API replies. But if you use MongoDB, try not to do this. It slows down your API.</p>
<p>Sometimes front-end team or other API clients ask to show them total number of objects in API replies. But if you use MongoDB, try not to do this. It slows down your API.</p>
<p>I say that if you have response format like that one placed below, you are in trouble.</p>
<div class="highlight"><pre><span></span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"total"</span><span class="p">:</span><span class="w"> </span><span class="err">XX</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"limit"</span><span class="p">:</span><span class="w"> </span><span class="err">ZZ</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"offset"</span><span class="p">:</span><span class="w"> </span><span class="err">YY</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"objects"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">]</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>If you have a big collection and quite heavy query, count will take prohibitively long time. Find may work fast, because of the limit. But count will scan entire result set.</p>
<p>Some times if you add some additional conditions, that should slow down query (e.g. regexp search), but it shrinks the result set, API may respond faster.</p>
<p>What is the solution?</p>
<p>First of all, if you can change API format, do it. Response with prev next links (like Facebook API does) can work faster.</p>
<div class="highlight"><pre><span></span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"pre"</span><span class="p">:</span><span class="w"> </span><span class="s2">"link-to-previous-x-objects"</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"next"</span><span class="p">:</span><span class="w"> </span><span class="s2">"link-to-next-x-objects"</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"limit"</span><span class="p">:</span><span class="w"> </span><span class="err">x</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"objects"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">]</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>Second option is to cache total. Then your response will look like search engines do. They can show you that they have more results than they actually have. If it's OK in your case, do like this.</p>
<p>In any case, it matters only for huge collections, heavy objects and complex queries. On the other cases you can do what you want without any problem.</p>
Don't use dicts so much2016-09-09T10:00:00+05:002016-09-09T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-09-09:/dict-vs-object.html<p>Python developers tend to use dicts too much. Mainly I tell about pass data through application as dicts not objects. It's a bad design witch causes many problems.</p>
<p>Python developers tend to use dicts too much. Mainly I tell about pass data through application as dicts not objects. It's a bad design witch causes many problems.</p>
<p>For example, our code calls external API and receives json (dict, actually). Should I pass this dict further or convert to the specific class? Both variants are possible, but I prefer to use classes and objects, although, it takes much more time and efforts to code.</p>
<p>This approach has some advantages. Such code is much easier to read. It's self-documented and maintainable.</p>
<p>Compare this:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">return_dict</span><span class="p">():</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">responses</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"some_api"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
</pre></div>
<p>And this:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">SomeObject</span><span class="p">:</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">from_json</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">json</span><span class="p">):</span>
<span class="n">instance</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">()</span>
<span class="o">//</span> <span class="n">do</span> <span class="n">magic</span>
<span class="k">return</span> <span class="n">instance</span>
<span class="k">def</span> <span class="nf">return_some_object</span><span class="p">():</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">responses</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"some_api"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">SomeObject</span><span class="o">.</span><span class="n">from_json</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">())</span>
</pre></div>
<p>What example is easier to read? What about maintainability?</p>
<p>I'm sure after a couple of months you forgot what return_dict returns. Event doc string doesn't help much. Response of such function validates very hard: usually such validation function is giant if-statement with many branches.</p>
<p>On the other hand, return_some_object return instance of the specific class. If you somewhere in your code receive such object, it's easy to discover what data it contains and what methods it provides. And if there is no documentation, it's quite easy to understand what this object is used for. In any case, it is much easier to deal with object of a specific class than dict.</p>
<p>Finally, code with classes much easier to maintain and test. Dict is to generic and say nothing about the data it contains. If dict contains nested dicts (with 99.99% probability), code maintenance becomes a nightmare.</p>
Move from Python to Java. First impressions2016-08-26T10:00:00+05:002016-08-26T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-08-26:/java-vs-python.html<p>I've added Java to my technology stack at work recently. We decided to do our new project using Spring Framework. For me it is a very interesting challenge, cause I've used Java only in my pet projects before. There are my very first impressions.</p>
<p>I've added Java to my technology stack at work recently. We decided to do our new project using Spring Framework. For me it is a very interesting challenge, cause I've used Java only in my pet projects before. There are my very first impressions.</p>
<p>Actually, I don't want to compare Python and Java much. Java is more verbose and has static types. It's obvious. I'm more interested in comparing mind sets and environments for both languages.</p>
<p>First of all, Java is more sensitive to quality of software architecture and design. Python is more flexible and forgive you some negligences in design before your code become mess. In contrast, Java loses readability dramatically.</p>
<p>Second, I was shocked how enterprise grade Java is different from Java for game dev or mobile. Language is the same, but frameworks for enterprise one is much more complex. Now I understand that Django is a tiny framework.</p>
<p>There are a lot of "magic" in Spring. It's very difficult to start. Luckily we have someone who has knowledge how Spring and related frameworks work.</p>
<p>Of course, there are fields where Java beats Python: it's much easier to deploy or it has advanced tools like Maven or Gradle.</p>
<p>In conclusion, I want to say that Java is'n better or worse than Python. It's different. And It manage you to change your programming habits:</p>
<ul class="simple">
<li>Change way to read code. In Java, method body less important to read than in Python, because a function declaration in Java more informative and solid. I don't say you shouldn't read method implementation. You should, but only then it is necessary.</li>
<li>Annotations in Java aren't equal to the decorators in Python. They look similar, but they work differently.</li>
<li>Java force you to use IDE feature more intensively. Relax. It's OK.</li>
<li>Read books. Effective Java must be in your top 3. Java frameworks like Spring much more complex and harder to understand than Python ones, but there are much higher quality materials to learn them.</li>
</ul>
<p>I'm not 100% Java developer now. About half of my time I'm still programming in Python. I will write posts about both languages Python and Java.</p>
How to use mypy on Python 2.7 project2016-07-29T10:00:00+05:002016-07-29T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-07-29:/mypy-python-27.html<p>If you work on a big Python project, type hinting can help you to find many design and function usage problems. Of course, you have to use external tools for static analysis, such as mypy. This is a great tool, but supports Python 3 mainly. Support for 2.7 branch isn't so straightforward. There are a couple of tricks that allows you to use it for 2.7.</p>
<p>If you work on a big Python project, type hinting can help you to find many design and function usage problems. Of course, you have to use external tools for static analysis, such as mypy. This is a great tool, but supports Python 3 mainly. Support for 2.7 branch isn't so straightforward. There are a couple of tricks that allows you to use it for 2.7.</p>
<div class="section" id="type-comments-vs-stub-files">
<h2>Type comments vs stub files</h2>
<p>First of all you should decide how to add type hints into your code. There are two options for Python 2.7: type comments or stub files. Personally I prefer type comments because it's easy to use and maintain.</p>
<p>A disadvantage of this approach is a necessity to import typing module. It looks quite strange. I only add comments, but mypy manages me to add an import statement into the code. Hopefully when we migrate to Python 3, we will change type comment to "in code" types, and that imports will become necessary.</p>
<p>Using stub files isn't an option for me. It's OK to use them for third party code. But it's really hard to manage ourselves to maintain them properly for our codebase.</p>
</div>
<div class="section" id="how-i-use-mypy">
<h2>How I use MyPy</h2>
<p>MyPy runs under Python 3.5. It means that we have to do several things before MyPy will check 2.7 code.</p>
<ul class="simple">
<li>Install python 3.5 somewhere in your system. Personally I prefer pyenv to do it. Make virtualenv and run <code>pip install mypy-lang</code> inside that environment.</li>
<li>go back to you python 2 project environment and install there back port of typing library: <code>pip install typing</code></li>
</ul>
<p>Check the installation. Activate mypy virtualenv and run <code>mypy --py2</code> inside the project root folder.</p>
</div>
<div class="section" id="integrate-with-pycharm">
<h2>Integrate with PyCharm</h2>
<p>I know that PyCharm already has a good type hinting system. But it works not only with PEP syntax, but with JetBrains own syntax. And it's not possible to use a PyCharm type checking in CI. Thats's why I integrate mypy checks into this IDE.</p>
<ul class="simple">
<li>Go Preferences -> Tools -> File Watchers. Add a new Watcher. Call it MyPy.</li>
<li>In Watcher Settings check Python file in File type field. Program should point to mypy executable inside virtualenv. Use this as Arguments: <cite>--py2 -s --check-untyped-defs $FilePath$</cite>. Press OK.</li>
</ul>
</div>
Writing code for Unity3d in Sublime Text 32016-07-22T10:00:00+05:002016-07-22T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-07-22:/unity3d-in-sublime.html<p>Using MonoDevelop IDE is the worst part of game developing on Unity. There is an alternative for Windows users. They can use Visual Studio. But should we, Mac users, do? Switch to Sublime Text.</p>
<p>Using MonoDevelop IDE is the worst part of game developing on Unity. There is an alternative for Windows users. They can use Visual Studio. But should we, Mac users, do? Switch to Sublime Text.</p>
<p>There is lack of information how doing it properly. I've spent a lot of time trying to add autocompletion for C# files in Sublime Text. Finally I found an easy way to do it. In fact, we need only 3 things installed on your Mac:</p>
<ul class="simple">
<li>git</li>
<li>mono (<code>brew install mono</code>)</li>
<li>Sublime Text</li>
</ul>
<p>Go to the folder with Sublime Text packages (Hint: use Preferences -> Browse Packages... menu item to locate the folder).</p>
<div class="highlight"><pre><span></span>git clone https://github.com/OmniSharp/omnisharp-sublime.git OmniSharp
<span class="nb">cd</span> OmniSharp/
git checkout roslyn
</pre></div>
<p>Add to sublime-project file link to solution file:</p>
<div class="highlight"><pre><span></span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"folders"</span><span class="p">:</span><span class="w"></span>
<span class="w"> </span><span class="p">[</span><span class="w"></span>
<span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"follow_symlinks"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"></span>
<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="s2">"."</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w"> </span><span class="p">],</span><span class="w"></span>
<span class="w"> </span><span class="nt">"solution_file"</span><span class="p">:</span><span class="w"> </span><span class="s2">"SolutionName.sln"</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>Restart Sublime Text. This is important. I wasted an hour trying to understand why this solution doesn't work.</p>
<p>That's all. Now you have autocomplete and code checker for your Unity game project.</p>
TrueType Fonts in LibGDX2016-07-14T10:00:00+05:002016-07-14T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-07-14:/libgdx-ttf.html<p>LibGDX is a beautiful library. It's easy to use. However some aspects of it isn't so clean. Dealing with fonts is an example of such aspects. I spent hours before I found reasonable solution that makes fonts looks good on an Android and on a desktop.</p>
<p>LibGDX is a beautiful library. It's easy to use. However some aspects of it isn't so clean. Dealing with fonts is an example of such aspects. I spent hours before I found reasonable solution that makes fonts looks good on an Android and on a desktop.</p>
<p>Forget about bitmap fonts. It takes a huge amount of time to fit sizes and prepare fixtures that look good on any device. I don't know any reason to do it. Use TrueType Fonts instead.</p>
<p>I know that using BitMapFont with distance field is <a class="reference external" href="https://github.com/libgdx/libgdx/wiki/Distance-field-fonts">recommended way</a>. But you have to do a lot of work before you get reasonable results. Reasonable but not perfect.</p>
<p>Using TTF gives a great result. There is one trick. You should use screen independent size for the font. There is the snippet in Kotlin how to prepare, convert, and add to skin TTF.</p>
<div class="highlight"><pre><span></span><span class="kd">class</span><span class="w"> </span><span class="nc">MyGame</span><span class="p">()</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">Game</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nv">skin</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">Skin</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Skin</span><span class="p">()</span><span class="w"></span>
<span class="w"> </span><span class="kd">override</span><span class="w"> </span><span class="kd">fun</span><span class="w"> </span><span class="nf">create</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">val</span><span class="w"> </span><span class="nv">generator</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">FreeTypeFontGenerator</span><span class="p">(</span><span class="n">Gdx</span><span class="p">.</span><span class="na">files</span><span class="p">.</span><span class="na">internal</span><span class="p">(</span><span class="s">"Roboto-Regular.ttf"</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="kd">val</span><span class="w"> </span><span class="nv">param</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">FreeTypeFontGenerator</span><span class="p">.</span><span class="na">FreeTypeFontParameter</span><span class="p">()</span><span class="w"></span>
<span class="w"> </span><span class="kd">val</span><span class="w"> </span><span class="nv">ratio</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Gdx</span><span class="p">.</span><span class="na">graphics</span><span class="p">.</span><span class="na">width</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="m">960f</span><span class="w"></span>
<span class="w"> </span><span class="n">param</span><span class="p">.</span><span class="na">size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="m">48</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">ratio</span><span class="p">).</span><span class="na">toInt</span><span class="p">()</span><span class="w"></span>
<span class="w"> </span><span class="kd">val</span><span class="w"> </span><span class="nv">robotoFont</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">generator</span><span class="p">.</span><span class="na">generateFont</span><span class="p">(</span><span class="n">param</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="n">skin</span><span class="p">.</span><span class="na">addRegions</span><span class="p">(</span><span class="n">TextureAtlas</span><span class="p">(</span><span class="n">Gdx</span><span class="p">.</span><span class="na">files</span><span class="p">.</span><span class="na">internal</span><span class="p">(</span><span class="s">"skin.atlas"</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="n">skin</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="s">"normaltext"</span><span class="p">,</span><span class="w"> </span><span class="n">robotoFont</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="n">skin</span><span class="p">.</span><span class="na">load</span><span class="p">(</span><span class="n">Gdx</span><span class="p">.</span><span class="na">files</span><span class="p">.</span><span class="na">internal</span><span class="p">(</span><span class="s">"skin.json"</span><span class="p">))</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w"> </span><span class="kd">override</span><span class="w"> </span><span class="kd">fun</span><span class="w"> </span><span class="nf">dispose</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="k">super</span><span class="p">.</span><span class="na">dispose</span><span class="p">()</span><span class="w"></span>
<span class="w"> </span><span class="n">skin</span><span class="p">.</span><span class="na">dispose</span><span class="p">()</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>As you see It's very easy. It works perfect. I don't notice any performance drops.</p>
PyCon Russia 20162016-07-07T10:00:00+05:002016-07-07T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-07-07:/pyconru-2016.html<p>PyConRu 2016 finished. It was the best PyConRu conference. Usually If there are two useful talks, conference is good. This time I found useful many amazing talks enough to fill an entire day-long session.</p>
<p>PyConRu 2016 finished. It was the best PyConRu conference. Usually If there are two useful talks, conference is good. This time I found useful many amazing talks enough to fill an entire day-long session.</p>
<p>It-People team did great work. They found beautiful place. They found great speakers and invited a lot interested people. Special thanks to Maria Ronacher. She managed all foreign speakers attended to the conference.</p>
<p>I hope there will be videos from the conference. I'll add them to this post as soon as organizers published them.</p>
Build own iterator over enumerate2016-06-27T17:00:00+05:002016-06-27T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-06-27:/logging-iterator.html<p>Couple days ago one of my colleagues asked me to implement simple self-logging iterator inherited from enumerate. I tried to inherit directly from it and failed. I absolutely forgot how to use <cite>__new__</cite> magic method. It was so embarrassing. I was in a hurry, so I promised myself to solve this puzzle. And I can tell you it is a simple task. Actually it takes 18 lines of code only.</p>
<p>Couple days ago one of my colleagues asked me to implement simple self-logging iterator inherited from enumerate. I tried to inherit directly from it and failed. I absolutely forgot how to use <cite>__new__</cite> magic method. It was so embarrassing. I was in a hurry, so I promised myself to solve this puzzle. And I can tell you it is a simple task. Actually it takes 18 lines of code only.</p>
<div class="section" id="initial-task">
<h2>Initial task</h2>
<p>First of all I have to explain why we need such iterator, why standard enumerate isn't enough.</p>
<p>In our project we had a lot of console tasks with similar structure:</p>
<ol class="arabic simple">
<li>Get bunch of object from DB</li>
<li>Log how many objects you have</li>
<li>Do something with each object from the bunch, log progress every X objects</li>
<li>Log after finish</li>
</ol>
<p>In python it looked like this:</p>
<div class="highlight"><pre><span></span><span class="n">iterable</span> <span class="o">=</span> <span class="n">get_bunch</span><span class="p">()</span>
<span class="n">total</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">iterable</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"total: </span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">total</span><span class="p">))</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">item</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">iterable</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">func</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"catch exception: </span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">e</span><span class="p">))</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">100</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"done </span><span class="si">{}</span><span class="s2"> of </span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">total</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Done!"</span><span class="p">)</span>
</pre></div>
<p>Difference between tasks was only in log messages and <em>func</em> function. It looked like copy-paste code. So we decided to refactor it.</p>
</div>
<div class="section" id="implementation">
<h2>Implementation</h2>
<p>Let's try to make class inherited from <cite>enumerate</cite>. As I mentioned above, we have to override <cite>__new__</cite> method because enumerate does it. You should remember that in accordance to documentation if <cite>__new__()</cite> returns an instance of cls, then the new instance’s <cite>__init__()</cite> method will be invoked with the same arguments.</p>
<p>So our implementation looks like these:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">LogEnumerate</span><span class="p">(</span><span class="nb">enumerate</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">iterable</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">LogEnumerate</span><span class="p">,</span> <span class="bp">cls</span><span class="p">)</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">iterable</span><span class="p">,</span> <span class="n">start</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">iterable</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span>
<span class="n">start_message</span><span class="o">=</span><span class="s1">''</span><span class="p">,</span> <span class="n">progress_message</span><span class="o">=</span><span class="s1">''</span><span class="p">,</span> <span class="n">stop_message</span><span class="o">=</span><span class="s1">''</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">progress_message</span> <span class="o">=</span> <span class="n">progress_message</span>
<span class="bp">self</span><span class="o">.</span><span class="n">stop_message</span> <span class="o">=</span> <span class="n">stop_message</span>
<span class="bp">self</span><span class="o">.</span><span class="n">step</span> <span class="o">=</span> <span class="n">step</span>
<span class="bp">self</span><span class="o">.</span><span class="n">total</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">iterable</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">start_message</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">start_message</span><span class="p">))</span>
<span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">i</span><span class="p">,</span> <span class="n">item</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__next__</span><span class="p">()</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">i</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">step</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">progress_message</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">total</span><span class="p">))</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">stop_message</span><span class="p">)</span>
<span class="k">raise</span>
</pre></div>
</div>
My top 7 of PyCon 2016 videos2016-06-14T17:00:00+05:002016-06-14T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-06-14:/pycon-2016-videos.html<p>There is my list with 7 best PyCon 2016 videos. I don't say that other videos not good. I say that these videos are more useful for me or impress me more than others. And as you can see I prefer talks about Python internals. I hope you'll find this list helpful.</p>
<p>There is my list with 7 best PyCon 2016 videos. I don't say that other videos not good. I say that these videos are more useful for me or impress me more than others. And as you can see I prefer talks about Python internals. I hope you'll find this list helpful.</p>
<h2>Larry Hastings - Removing Python's GIL: The Gilectomy</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/P3AyI_u66Bw" frameborder="0" allowfullscreen></iframe>
<h2>Andrey Petrov - See Python, See Python Go, Go Python Go</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/CkDwb5koRTc" frameborder="0" allowfullscreen></iframe>
<h2>Chelsea Voss - Oneliner-izer: An Exercise in Constrained Coding</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/DsUxuz_Rt8g" frameborder="0" allowfullscreen></iframe>
<h2>Russell Keith-Magee - A tale of two cellphones</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/NqdpK9KjGgQ" frameborder="0" allowfullscreen></iframe>
<h2>Glyph - Shipping Software To Users With Python</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/5BqAeN-F9Qs" frameborder="0" allowfullscreen></iframe>
<h2>Scott Sanderson, Joe Jevnik - Playing with Python Bytecode</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/mxjv9KqzwjI" frameborder="0" allowfullscreen></iframe>
<h2>Matthias Kramm - Python Typology</h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/IDm_YIQihhs" frameborder="0" allowfullscreen></iframe>Mutable types as default params in Python2016-06-09T10:00:00+05:002016-06-09T10:00:00+05:00akorablevtag:www.alexkorablev.com,2016-06-09:/mutable-types-as-default-params-in-python.html<p>Do you ask yourself why you souldn't use mutable types as default value for parameters in Python? I'm sure you ask at least if you are not a junior. Probably, your answer is "It causes to strange side effects". However, only a tiny part of python developers makes step throw and figure out how it works under the hood.</p>
<p>Do you ask yourself why you souldn't use mutable types as default value for parameters in Python? I'm sure you ask at least if you are not a junior. Probably, your answer is "It causes to strange side effects". However, only a tiny part of python developers makes step throw and figure out how it works under the hood.</p>
<p>At the beginning, we should find out what kind of side effects we deal with. Look at this example:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">param</span><span class="o">=</span><span class="p">[]):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">param</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[]</span>
</code></pre></div>
<p>There are no side effects in this function. It's good. That's what we expect. But if we modify it this way:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">param</span><span class="o">=</span><span class="p">[]):</span>
<span class="n">param</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">param</span><span class="p">)</span>
</code></pre></div>
<p>We will find that function prints different values depends on how many times we use it.</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div>
<p>We don't want this. Such function behaviour in Python is the leader of "Top strange things of Python" lists. Although if you make some efforts to figure out how Python acts under the hood, this "strange" behaviour will become consistent and logical.</p>
<p>In Python function is object like all other things in that language. Then it creates function object, it packs default param values into foo.func_defaults tuple. </p>
<p>List in Python is mutable. Python makes function object ones. This results that object foo.func_defaults[0] and param inside function point to the same object in memory.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">param</span><span class="o">=</span><span class="p">[]):</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">id</span><span class="p">(</span><span class="n">param</span><span class="p">))</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="mi">4494207600</span>
<span class="o">>>></span> <span class="nb">id</span><span class="p">(</span><span class="n">foo</span><span class="o">.</span><span class="n">func_defaults</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="mi">4494207600</span>
</code></pre></div>
<p>You should use None as default value instead list, dict or any other mutable types and initialize mutable object inside function body. To hint type use docstring or <a href="https://www.python.org/dev/peps/pep-0484/">PEP 0484</a> if you use Python 3.5+.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">param</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="sd">"""</span>
<span class="sd"> :type param: list</span>
<span class="sd"> """</span>
<span class="n">param</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">if</span> <span class="n">param</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">param</span>
<span class="n">param</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">param</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">foo</span><span class="p">()</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div>
<p>For short functions adding this kind of checks makes them less readable. In such cases you should decide to leave this code and make a function safer or remove it. I choose safety. You can miss changes that add side effects doing refactoring later.</p>Kivy Project. Part 1. Setting up environment2016-06-01T17:00:00+05:002016-06-01T17:00:00+05:00akorablevtag:www.alexkorablev.com,2016-06-01:/setup-kivy-project.html<p><a href="https://kivy.org/#home>">Kivy</a> is an interesting project that allows you build GUI applications for different platform including iOS and Android. If you believe the rumors... I decided to check out is Kivy suits to build for small python GUI app for 5 major platforms: Mac OS X, Linux, Windows, iOS, and Android. In this series I'll try to build a small app for learning foreign language vocabulary and deploy it on all these platforms.</p>
<p><a href="https://kivy.org/#home>">Kivy</a> is an interesting project that allows you build GUI applications for different platform including iOS and Android. If you believe the rumors... I decided to check out is Kivy suits to build for small python GUI app for 5 major platforms: Mac OS X, Linux, Windows, iOS, and Android. In this series I'll try to build a small app for learning foreign language vocabulary and deploy it on all these platforms.</p>
<p>This series woun't be a tutorial. It'll be more like a log of my tries and failures. I'll cover very basic things. No street magic. If everything goes as I expect, at the end I'll get 3 or 4 executables.</p>
<p><em>NB</em>: I use primary Mac for development. That's why most instruction will be for that platform. May be later I'll add instructions for Linux and Windows.</p>
<h2>Set up Kivy</h2>
<p>There are two ways to set up Kivy on Mac. You can download and setup bundle from homebrew. Or you can install Kivy from PyPi.</p>
<h3>First way (fastest):</h3>
<ul>
<li>Download from http://kivy.org/#download Kivy2.7z (if you prefer Python 2.7) or Kivy3.7z (includes Python 3.5)</li>
<li>Extract it</li>
<li>Copy file to Application</li>
</ul>
<div class="highlight"><pre><span></span><code>sudo mv Kivy3.app /Applications/Kivy.app
</code></pre></div>
<ul>
<li>Make symlink</li>
</ul>
<div class="highlight"><pre><span></span><code>ln -s /Applications/Kivy.app/Contents/Resources/script /usr/local/bin/kivy
</code></pre></div>
<p>This approach has a disadvantage. Kivy.app contains only one virtualenv for all projects. It's OK for small, simple projects, but it can cause problems for big ones. That's why I suppose to install framework in a separate environment for each project.</p>
<h3>Second way (the rith one):</h3>
<p>I assume that you use <a href="https://github.com/yyuu/pyenv">pyenv</a> and <a href="https://github.com/yyuu/pyenv-virtualenv">pyenv-virtualenv</a>.</p>
<p>Install dependencies first.</p>
<div class="highlight"><pre><span></span><code>brew install sdl2 sdl2_image sdl2_ttf sdl2_mixer gstreamer
</code></pre></div>
<p>Install latest Python.</p>
<div class="highlight"><pre><span></span><code>pyenv install <span class="m">3</span>.5.1
</code></pre></div>
<p>Some times it fails:</p>
<div class="highlight"><pre><span></span><code>zipimport.ZipImportError: can<span class="err">'</span>t decompress data<span class="p">;</span> zlib not available
</code></pre></div>
<p>In that case this command helps:</p>
<div class="highlight"><pre><span></span><code>xcode-select --install
</code></pre></div>
<p>As soon as we get python installed, make virtualenv for our project.</p>
<div class="highlight"><pre><span></span><code>pyenv virtualenv <span class="m">3</span>.5.1 WordConstructor
</code></pre></div>
<p>Activate it.</p>
<div class="highlight"><pre><span></span><code>pyenv activate WordConstructor
</code></pre></div>
<p>Install Cython inside virtualenv.</p>
<div class="highlight"><pre><span></span><code>pip install -I <span class="nv">Cython</span><span class="o">==</span><span class="m">0</span>.23
</code></pre></div>
<p>Install Kivy.</p>
<div class="highlight"><pre><span></span><code><span class="nv">USE_OSX_FRAMEWORKS</span><span class="o">=</span><span class="m">0</span> pip install kivy
</code></pre></div>
<p>Let's check that everything is working. Open new file main.py in the project folder. Fill it with that content:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">kivy.app</span> <span class="kn">import</span> <span class="n">App</span>
<span class="kn">from</span> <span class="nn">kivy.uix.widget</span> <span class="kn">import</span> <span class="n">Widget</span>
<span class="k">class</span> <span class="nc">WordConstructorGame</span><span class="p">(</span><span class="n">Widget</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">WordConstructorApp</span><span class="p">(</span><span class="n">App</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="n">WordConstructorGame</span><span class="p">()</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">WordConstructorApp</span><span class="p">()</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>Save and run it.</p>
<div class="highlight"><pre><span></span><code>python main.py
</code></pre></div>
<p>If a window with title "WordConstructor" appeared, everything works OK.</p>
<h2>Tuning PyCharm</h2>
<p>Let's set up PyCharm project.</p>
<p>In the Preferences dialog, choose Project interpreter corresponds our virtualenv.</p>
<p><img alt="Kivy interpreter" src="https://www.alexkorablev.com/images/kivy-interpreter.png"></p>
<p>Kivy uses Kv Design Language to describe UI. It would be nice to have syntax highlight and autocomplit for it. </p>
<ol>
<li>Download https://github.com/Zen-CODE/kivybits/blob/master/IDE/PyCharm_kv_completion.jar?raw=true</li>
<li>Import it in menu File -> Import Settings in PyCharm.</li>
<li>Check File types on and press OK.</li>
<li>Restart PyCharm. Done. </li>
</ol>
<p>That's all about setting up the environment for Kivy. Second part will cover more practical topics.</p>