tag:blogger.com,1999:blog-41951352461071662512024-03-12T20:51:17.570-07:00NeopythonicRamblings through technology, politics, culture and philosophy by the creator of the Python programming language.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.comBlogger46125tag:blogger.com,1999:blog-4195135246107166251.post-53775406319713416452022-10-04T23:39:00.000-07:002022-10-04T23:39:50.596-07:00Reasoning about asyncio.Semaphore<p> In Silicon Valley is a very exclusive fast-food restaurant, which is always open. There is one table, where one guest at a time is served an absolutely fabulous hamburger. When you arrive, you wait in line until the table is available. Then the host takes you to the table and, this being America, you are asked a seemingly endless series of questions about how you would like your hamburger to be cooked and served.</p><p>But today we're not talking about culinary delights. We're talking about the queuing system used by the restaurant. If you are lucky to arrive at the restaurant when the table is available and there are no other guests waiting, you are seated right away. Otherwise, the host gives you a buzzer (from an infinite stack of buzzers!) and you are free to roam the neighborhood until your buzzer goes off. It is the host's job to ensure that guests are seated in order of arrival. When it is your turn, the host will cause your buzzer go off and you make your way back to the restaurant, where you will be seated.</p><p>If you change your mind, you can return the buzzer to the host, who will take it back without lifting an eyebrow. If your buzzer has already gone off, the host will buzz the next guest, if any. Guests are always polite and don't abscond with their buzzers. The host is always fair and doesn't seat another guest ahead of you even if you take your time making it back.<br /></p><p>The above description fits that of a <a href="https://docs.python.org/3.12/library/asyncio-sync.html#asyncio.Lock" target="_blank">Lock</a>. A guest arriving corresponds to the <i>acquire()</i> call; leaving is a <i>release()</i> call. Changing your mind is like getting cancelled while waiting in <i>acquire()</i>. You can change your mind before or after your buzzer goes off, i.e., you can be cancelled before or after the lock has awakened your call (but before you return from <i>acquire()</i>).</p><p>One day the restaurant expands, hiring extra sous-chefs and opening several new tables. There is still only one host, whose job is not really changed. However, since multiple guests can be seated concurrently, a <a href="https://docs.python.org/3.12/library/asyncio-sync.html#asyncio.Semaphore" target="_blank">Semaphore</a> must now be used instead of a simple Lock.<br /></p><p>It turns out that implementing synchronization primitives is hard. This is somewhat surprising in the case of <i>asyncio</i>, since only one task can be executing at a time, and task switches only happen at <i>await</i>. But in the past year its <a href="https://github.com/python/cpython/issues/90155" target="_blank">fairness</a>, <a href="https://github.com/python/cpython/issues/90155#issuecomment-1247764412" target="_blank">correctness</a>, <a href="https://github.com/python/cpython/issues/97028" target="_blank">semantics</a> and <a href="https://github.com/python/cpython/issues/97545" target="_blank">performance</a> have all been challenged. In fact, the last three complaints happened in the last month, and, being the last <a href="https://github.com/python/cpython/labels/expert-asyncio" target="_blank"><i>asyncio</i> expert</a> standing, I had to learn in a hurry what's the best way to think about semaphores.</p><p>The restaurant metaphor was very useful. For example, there is a difference between the number of open tables and the number of guests who may be seated immediately, and it equals the number of guests whose buzzer has gone off but who haven't come back to the host yet.</p><p>There was one particular challenge to fairness, where a task that released a semaphore and then immediately tried to acquire it again could starve other tasks. This is like a guest walking out, turning around, and getting seated again ahead of other waiting guests.</p><p>And there was a bug where a cancelled <i>acquire()</i> call could leave the lock in a bad state. This is like the host getting confused when a guest with a buzzing buzzer returns it but declines to be seated.</p><p>The restaurant metaphor didn't help with everything: cancellation behavior in <i>asyncio</i> is just complex. In Python 3.11 we have started putting extra strain on cancellation, because of two new asynchronous context managers we added:</p><ul style="text-align: left;"><li>Class <a href="https://docs.python.org/3.12/library/asyncio-task.html#task-groups" target="_blank">TaskGroup</a> for managing a group of related tasks. When one task fails, the others are cancelled, and the context manager waits for all tasks to exit.</li><li><a href="https://docs.python.org/3.12/library/asyncio-task.html#asyncio.timeout" target="_blank">timeout()</a> function for managing timeouts. When the timeout goes off, the current task is cancelled.</li></ul><p>Here is the main complication of cancellation handling: <br /></p><ul style="text-align: left;"><li>When waiting for a <a href="https://docs.python.org/3.12/library/asyncio-future.html#asyncio.Future" target="_blank">Future</a>, that Future may be cancelled, and then the <i>await</i> operation fails, raising <a href="https://docs.python.org/3.12/library/asyncio-exceptions.html#asyncio.CancelledError" target="_blank">CancelledError</a>.</li><li>But when awaiting a Future raises CancelledError you cannot assume that the Future was cancelled! It is also possible that the Future was already marked as having a result (so it can no longer be cancelled), and your task has been marked as runnable, but another (also runnable) task runs first and cancels your task. I am grateful to Cyker Way for pointing out this corner case.</li></ul><p> It helps to think of Futures as being in one of four states:</p><ul style="text-align: left;"><li>Waiting</li><li>Done, holding a result</li><li>Done, holding an exception</li><li>Done, but cancelled</li></ul><p>From the waiting state a Future can transition to one of the other states, and then it cannot change state again. (Insert cute picture of state diagram here. :-)<br /></p><p>The semaphore manages a FIFO queue of waiters. It does not use the exception state, but it does use the other three states:</p><ul style="text-align: left;"><li>Waiting: a guest with a buzzer that hasn't gone off yet</li><li>Holding a result: a guest who has been buzzed</li><li>Cancelled: a guest who returns their buzzer before it goes off</li></ul><p>Fairness is supposed to be ensured by always appending a new Future to the queue to the end when <i>acquire()</i> finds the semaphore locked, and by always marking the leftmost (i.e., oldest) Future in the queue as holding a result when <i>release()</i> is called while queue isn't empty. The fairness bug was due <i>acquire()</i> taking a shortcut when the Semaphore's <i>level</i> (the number of open tables) is nonzero. It should not do this when there are still Futures in the queue. In other words we were sometimes seating a newly arrived guest when there was an open table even though there was already a guest waiting.<br /></p><p>Guess what caused the cancellation bug? The scenario where a Future is holding a result (guest with buzzer buzzing) but the task awaiting that Future gets cancelled (guest declining to be seated).</p><p>I struggled to visualize the state of the Semaphore for myself, with its <i>level</i> and <i>FIFO queue</i> of waiting Futures. I also struggled with the definition of <i>locked()</i>. If the <i>level</i> variable had been public I would have struggled with its semantics too. In the end I came up with the following definitions:</p><ul style="text-align: left;"><li>W, the list of waiting futures, or [<i>f</i> for <i>f</i> in <i>queue</i> if not <i>f.done()</i>]</li><li>R, the list of futures holding a result, or [<i>f</i> for <i>f</i> in <i>queue</i> if <i>f.done()</i> and not <i>f.cancelled()</i>]</li><li>C, the list of cancelled futures, or [<i>f</i> for <i>f</i> in queue if <i>f.cancelled()</i>]</li></ul><p> and some invariants:</p><ul style="text-align: left;"><li>set(W + R + C) == set(<i>queue</i>) — all futures are either waiting, have a result, or are cancelled.<br /></li><li><i>level</i> >= len(R) — we must have at least as many open tables as there are guests holding buzzing buzzers.<br /></li><li>define <i>locked()</i> as (len(W) > 0 or len(R) > 0 or <i>level</i> == 0) — we cannot immediately seat anyone unless (a) there are no guests waiting for their buzzer to go off, (b) there are no guests holding a buzzing buzzer, and (c) there is at least one open table.</li></ul><p> I leave you with a final link, to the <a href="https://github.com/python/cpython/blob/c70c8b69762f720377adaf22f2e5ec6496a7be53/Lib/asyncio/locks.py#L330" target="_blank">current code</a>.<br /></p>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-84682757276949914412022-02-28T22:19:00.004-08:002022-03-01T00:03:45.375-08:00Meeting Mike Burrows<p> </p><div class="Ar Au Ao" id=":1u0" style="display: block;"><div aria-label="Message Body" aria-multiline="true" class="Am Al editable LW-avf tS-tW tS-tY" contenteditable="true" id=":2fh" role="textbox" spellcheck="false" style="direction: ltr; min-height: 226px;" tabindex="1"><div>In late 2005 I joined Google. The interviews took a surprising long time, which is a tale for another time. Today I want to tell a story that happened in one of my first weeks on campus.</div><div><br /></div><div>In the main building was an impressive staircase going up to the second floor. Somewhere near the top was a spacious office. A very important engineer worked there. I checked the name on the door and realized I knew him: he had been a grad student from the UK who had spent some time visiting our research group (the Amoeba project) at CWI in Amsterdam in the early '90s.<br /></div><div><br /></div><div>Happy to find someone I knew long ago, one day I knocked on the door and introduced myself. Yes, he remembered me too, but my delight was soon over. Not only was Python the bane of Mike's existence at Google (he detested everything that wasn't C++), but the one memory from his stay in Amsterdam that stood out was about a time I had given him a ride across town on the back of my bike: "Worst ride of my life."</div></div></div>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-7753394721732539222019-03-15T10:58:00.001-07:002019-03-15T14:38:36.600-07:00Why operators are usefulThis is something I posted on python-ideas, but I think it's interesting to a wider audience.<br />
<br />
There's been a lot of discussion recently about an operator to merge two dicts.<br />
<br />
It prompted me to think about the reason (some) people like operators, and a discussion I had with my mentor Lambert Meertens over 30 years ago came to mind.<br />
<br />
For mathematicians, operators are essential to how they think. Take a simple operation like adding two numbers, and try exploring some of its behavior.<br />
<br />
add(x, y) == add(y, x) (1)<br />
<br />
Equation (1) expresses the law that addition is commutative. It's usually written using an operator, which makes it more concise:<br />
<br />
x + y == y + x (1a)<br />
<br />
That feels like a minor gain.<br />
<br />
Now consider the associative law:<br />
<br />
add(x, add(y, z)) == add(add(x, y), z) (2)<br />
<br />
Equation (2) can be rewritten using operators:<br />
<br />
x + (y + z) == (x + y) + z (2a)<br />
<br />
This is much less confusing than (2), and leads to the observation that the parentheses are redundant, so now we can write<br />
<br />
x + y + z (3)<br />
<br />
without ambiguity (it doesn't matter whether the + operator binds tighter to the left or to the right).<br />
<br />
Many other laws are also written more easily using operators. Here's one more example, about the identity element of addition:<br />
<br />
add(x, 0) == add(0, x) == x (4)<br />
<br />
compare to<br />
<br />
x + 0 == 0 + x == x (4a)<br />
<br />
The general idea here is that once you've learned this simple notation, equations written using them are easier to *manipulate* than equations written using functional notation -- it is as if our brains grasp the operators using different brain machinery, and this is more efficient.<br />
<br />
I think that the fact that formulas written using operators are more easily processed *visually* has something to do with it: they engage the brain's visual processing machinery, which operates largely subconsciously, and tells the conscious part what it sees (e.g. "chair" rather than "pieces of wood joined together"). The functional notation must take a different path through our brain, which is less subconscious (it's related to reading and understanding what you read, which is learned/trained at a much later age than visual processing).<br />
<br />
The power of visual processing really becomes apparent when you combine multiple operators. For example, consider the distributive law:<br />
<br />
mul(n, add(x, y)) == add(mul(n, x), mul(n, y)) (5)<br />
<br />
That was painful to write, and I believe that at first you won't see the pattern (or at least you wouldn't have immediately seen it if I hadn't mentioned this was the distributive law).<br />
<br />
Compare to:<br />
<br />
n * (x + y) == n * x + n * y (5a)<br />
<br />
Notice how this also uses relative operator priorities. Often mathematicians write this even more compact:<br />
<br />
n(x+y) == nx + ny (5b)<br />
<br />
but alas, that currently goes beyond the capacities of Python's parser.<br />
<br />
Another very powerful aspect of operator notation is that it is convenient to apply them to objects of different types. For example, laws (1) through (5) also work when x, y and z are same-size vectors and n is a scalar (substituting a vector of zeros for the literal "0"), and also if they are matrices (again, n has to be a scalar).<br />
<br />
And you can do this with objects in many different domains. For example, the above laws (1) through (5) apply to functions too (n being a scalar again).<br />
<br />
By choosing the operators wisely, mathematicians can employ their visual brain to help them do math better: they'll discover new interesting laws sooner because sometimes the symbols on the blackboard just jump at you and suggest a path to an elusive proof.<br />
<br />
Now, programming isn't exactly the same activity as math, but we all know that Readability Counts, and this is where operator overloading in Python comes in. Once you've internalized the simple properties which operators tend to have, using + for string or list concatenation becomes more readable than a pure OO notation, and (2) and (3) above explain (in part) why that is.<br />
<br />
Of course, it's definitely possible to overdo this -- then you get Perl. But I think that the folks who point out "there is already a way to do this" are missing the point that it really is easier to grasp the meaning of this:<br />
<br />
d = d1 + d2<br />
<br />
compared to this:<br />
<br />
d = d1.copy()<br />
d.update(d2) # CORRECTED: This line was previously wrong<br />
<br />
and it is not just a matter of fewer lines of code: the first form allows us to use our visual processing to help us see the meaning quicker -- and without distracting other parts of our brain (which might already be occupied by keeping track of the meaning of d1 and d2, for example).<br />
<br />
Of course, everything comes at a price. You have to learn the operators, and you have to learn their properties when applied to different object types. (This is true in math too -- for numbers, x*y == y*x, but this property does not apply to functions or matrices; OTOH x+y == y+x applies to all, as does the associative law.)<br />
<br />
"But what about performance?" I hear you ask. Good question. IMO, readability comes first, performance second. And in the basic example (d = d1 + d2) there is no performance loss compared to the two-line version using update, and a clear win in readability. I can think of many situations where performance difference is irrelevant but readability is of utmost importance, and for me this is the default assumption (even at Dropbox -- our most performance critical code has already been rewritten in ugly Python or in Go). For the few cases where performance concerns are paramount, it's easy to transform the operator version to something else -- *once you've confirmed it's needed* (probably by profiling).Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-24711469724337158072018-11-26T09:13:00.001-08:002018-11-28T20:16:59.301-08:00What to do with your computer science careerI regularly receive questions from students in the field of computer science looking for career advice.<br />
<br />
Here's an answer I wrote to one of them. It's not comprehensive or anything, but I thought people might find it interesting.<br />
<br />
[A question about whether to choose a 9-5 job or be an entrepreneur]<br />
<br />
The question about "9-5" vs. "entrepreneur" is a complex one -- not everybody can be a successful entrepreneur (who would do the work? :-) and not everybody has the temperament for it. For me personally it was never an option -- there are vast parts of management and entrepreneurship that I wouldn't enjoy doing, such as hiring (I hate interviewing and am bad at it) and firing (too emotionally draining -- even just giving negative feedback is hard for me). Pitching ideas to investors is another thing that I'd rather do without.<br />
<br />
If any of that resonates with you, you may be better off not opting for entrepreneurship -- the kind of 9-5 software development jobs I have had are actually (mostly) very rewarding: I get to write software that gets used by hundreds or thousands of other developers (or millions in the case of Python), and those other developers in turn use my software to produce product that get uses by hundreds of thousands or, indeed hundreds of millions of users. Not every 9-5 job is the same! For me personally, I don't like the product stuff (since usually that means it's products I have no interest in using myself), but "your mileage may vary" (as they say in the US). Just try to do better than an entry-level web development job; that particular field (editing HTML and CSS) is likely to be automated away, and would feel repetitive to me.<br />
<br />
[A question about whether AI would make human software developers redundant (not about what I think of the field of AI as a career choice)]<br />
<br />
Regarding AI, I'm not worried at all. The field is focused on automating boring, repetitive tasks like driving a car or recognizing faces, which humans can learn to do easily but find boring if they have to do it all the time. The field of software engineering (which includes the field of AI) is never boring, since as soon as a task is repetitive, you automate it, and you start solving new problems.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-24681072269625122882016-07-23T14:11:00.004-07:002016-07-23T14:11:39.044-07:00About spammers and commentsI'm turning off commenting for my blogs. While I've enjoyed some feedback, the time wasted to moderate spam posts just isn't worth it. Thank you, spammers! :-(Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-43871756086799248412016-05-18T11:55:00.000-07:002016-05-18T11:55:02.997-07:00Union syntax<h2>
Union syntax</h2>
<blockquote class="tr_bq">
<i>(I'm trying to do this as a quick post in response to some questions I received on this topic. I realize this will probably reopen the whole discussion about the best syntax for types, but sorry folks, PEP 484 was accepted nearly a year ago, after many months of discussions and hundreds of messages. It's unlikely that any idea you can think of here would be new. This post just explains the rationale of one particular decision and tries to put it in some context.)</i></blockquote>
I've heard some grumbling about the union syntax in <a href="https://www.python.org/dev/peps/pep-0484/">PEP 484</a>: Union[X, Y, Z] (where X, Y and Z are arbitrary type expressions). In the past people have suggested X|Y|Z for this, or (X, Y, Z) or {X, Y, Z}. Why did we go with the admittedly clunkier Union[X, Y, Z]?<br />
<br />
First of all, despite all the attention drawn to it, unions are actually a pretty minor feature, and you shouldn't be using them much. So you also shouldn't care that much.<br />
<h3>
Why not X|Y|Z?</h3>
This won't fly because we want compatibility with versions of Python 3 that were already frozen (see below). We want to be able to express e.g. a union of int and str, which under this notation would be written as int|str. But for that to fly we'd have to modify the builtin 'type' class to implement __or__ -- and that wouldn't fly on already-frozen Python versions. Supporting X|Y only for types (like List) imported from the typing module and some other notation for builtin types would only sow confusion. So X|Y|Z is out.<br />
<h3>
Why not {X, Y, Z}?</h3>
That's the set with elements X, Y and Z, using the builtin set notation. We can usefully consider types to be sets of values, and this makes a union a set of values too (that's why it's called union :-).<br />
<br />
However, {X, Y, Z} confuses the set of <i>types</i> with the set of <i>values</i>, which I consider a mortal sin. This would just cause endless confusion.<br />
<br />
This notation would also confuse things when taking the union of several classes that overlap, e.g. if we have classes B and C, where C inherits from B, then the union of B and C is just B. But the builtin set doesn't see it that way. In contrast, the X|Y notation could actually solve this (since in principle we could overload __or__ to do whatever we want), and the Union[] operator ("functor"?) from PEP 484 indeed solves this -- in this example Union[B, C] returns the (non-union) type B, both in the type checker and at runtime.<br />
<h3>
Why not (X, Y, Z)?</h3>
That's the tuple (X, Y, Z). It has the same disadvantages as {X, Y, Z}, but at least it has the advantage of being similar to how unions are expressed as arguments to isinstance(), for example isinstance(x, (int, str, list)) or isinstance(x, (Sequence, Mapping)). (Similarly the except clause: try: ... / except (KeyError, IndexError): ...)<br />
<br />
Another problem with tuples is that the tuple syntax is already overloaded in so many ways that it would be confused with other uses even more easily. One particular confusion would be other generic types, for which we'd still want to use square brackets. (You can't really beat Iterable[int] for clarity if you have an iterable of integers. :-) Suppose you have a sequence of values that could be integers or strings. In PEP 484 notation we write this as Sequence[Union[int, str]]. Using the tuple notation we'd want to write this as Sequence[(int, str)]. But it turns out that the __getitem__ overload on the metaclass can't tell the difference between Sequence[(int, str)] and Sequence[int, str] -- and we would like to reject the latter as a mistake since Sequence[] is a generic class over a single parameter. (An example of a generic class over two parameters would be Mapping[K, V].) Disambiguating all this would place us on very thin ice indeed.<br />
<br />
The nail in this idea's coffin is the competing idea of using (X, Y, Z) to indicate a tuple with three items, with respective types, X, Y and Z. At first sight this seems an even better use of the tuple syntax than unions would be, and tuples are way more common than unions. But it runs afoul of the same problems with Foo[(X, Y)] vs. Foo[X, Y]. (Also, there would be no easy way to describe what PEP 484 calls Tuple[X, ...], i.e. a variable-length tuple with uniform item type X.)<br />
<h3>
PS. Why support old Python 3 versions?</h3>
The reason for supporting older versions is adoption. Only a relatively small crowd of early adopters can upgrade to the latest Python version as soon as it's out; the rest of us are stuck on older versions (even Python 2.7!). <br />
<br />
So for PEP 484 and the typing module, we wanted to support 3.2 and up -- we chose 3.2 because it's the newest Python 3 supported by some older but still popular Ubuntu and Debian distributions. (Also, 3.0 and 3.1 were too immature at their time of release to ever have a large following.)<br />
<br />
There's a typing package that you can install easily using pip, and this defines all sorts of useful things for typing, from Any and Union to generic versions of List and Sequence. But such a package can't modify existing builtins like int or list.<br />
<br />
(Eventually we also added Python 2.7 support, using type comments for function signatures.)Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com0tag:blogger.com,1999:blog-4195135246107166251.post-88541851060459732132016-05-18T07:06:00.001-07:002016-05-18T08:39:37.922-07:00Adding type annotations for fspath<div>
<h1 class="ace-copy-paste-skip-this-tag">
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Type annotations for fspath</span></h1>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Python 3.6 will have a new </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://www.pixelmonkey.org/2013/04/11/python-double-under-double-wonder" href="http://www.pixelmonkey.org/2013/04/11/python-double-under-double-wonder" rel="noreferrer nofollow" target="_blank">dunder protocol</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which should be supported by classes that represent filesystem paths. Example of such classes are the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">pathlib.Path</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> family and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> (returned by </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ).</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">You can read more about this protocol in the brand new </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0519/" href="https://www.python.org/dev/peps/pep-0519/" rel="noreferrer nofollow" target="_blank">PEP 519</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. In this blog post I’m going to discuss how we would add type annotations for these additions to the standard library.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">I’m making frequent use of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , a quite magical type variable predefined in the typing module. If you’re not familiar with it, I recommend reading my </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://neopythonic.blogspot.com/2016/05/the-anystr-type-variable.html" href="http://neopythonic.blogspot.com/2016/05/the-anystr-type-variable.html" rel="noreferrer nofollow" target="_blank">blog post about </a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code attrlink url"><a class="attrlink" data-target-href="http://neopythonic.blogspot.com/2016/05/the-anystr-type-variable.html" href="http://neopythonic.blogspot.com/2016/05/the-anystr-type-variable.html" rel="noreferrer nofollow" target="_blank">AnyStr</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . You may also want to read up on </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0484/#generics" href="https://www.python.org/dev/peps/pep-0484/#generics" rel="noreferrer nofollow" target="_blank">generics in PEP 484</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> (or read </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://mypy.readthedocs.io/en/latest/generics.html" href="http://mypy.readthedocs.io/en/latest/generics.html" rel="noreferrer nofollow" target="_blank">mypy’s docs on the subject</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">).</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Adding os.scandir() to the stubs for os.py</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">For practice, let’s see if we can add something to the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" rel="noreferrer nofollow" target="_blank">stub file for os.py</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. As of this writing there’s no </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed" href="https://github.com/python/typeshed" rel="noreferrer nofollow" target="_blank">typeshed</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> information for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code attrlink url"><a class="attrlink" data-target-href="https://docs.python.org/3/library/os.html" href="https://docs.python.org/3/library/os.html" rel="noreferrer nofollow" target="_blank">os.scandir()</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which I think is a shame. I think the following will do nicely. Note how we only define </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> for Python versions >= 3.5. (Mypy </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/mypy/issues/698" href="https://github.com/python/mypy/issues/698" rel="noreferrer nofollow" target="_blank">doesn’t support this yet</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, but it will soon, and the example here still works — it just doesn’t realize </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is only available in Python 3.5.) This could be added to the end of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">stdlib/3/os/__init__.pyi</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import Generic, AnyStr, overload, Iterator</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">if sys.version_info </span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span>= (3, 5):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> class DirEntry(</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-42889384956">Generic</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">[AnyStr]):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> name = ... # type: AnyStr</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> path = ... # type: AnyStr</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def inode(self) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> int: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def is_dir(self, *, follow_symlinks: bool = ...) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> bool: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def is_file(self, *, follow_symlinks: bool = ...) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> bool: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def is_symlink(self) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> bool: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def stat(self, *, follow_symlinks: bool = ...) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> stat_result: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> @overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> def scandir() -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> @overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> def scandir(path: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-21653656371 thread-53340393283">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[AnyStr]]: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Deconstructing this a bit, we see a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0484/#generics" href="https://www.python.org/dev/peps/pep-0484/#generics" rel="noreferrer nofollow" target="_blank">generic class</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> (that’s what the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Generic[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> base class means) and an overloaded function. The </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> definition uses </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">@overload</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> because it can also be called without arguments. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">We could also write it as follows; it’ll work either way:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> @overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"> def scandir(</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">path: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-72143937476">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> = ...) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> @overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def scandir(path: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-86865838424">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">]]: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Either way there really are three ways to call </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir()</span><span class=""> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, all three returning an iterable of DirEntry objects:</span></div>
<div>
<br /></div>
<ul class="listtype-bullet listindent1 list-bullet1">
<li><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir() -</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span></li>
<li><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir(str) -</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span></li>
<li><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir(bytes) -</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[bytes]]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span></li>
</ul>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Adding os.fspath()</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Next I’ll show how to add </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.fspath()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> and how to add support for the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> protocol to </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> .</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0519/" href="https://www.python.org/dev/peps/pep-0519/" rel="noreferrer nofollow" target="_blank">PEP 519</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> defines a simple ABC (</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://docs.python.org/3/library/abc.html" href="https://docs.python.org/3/library/abc.html" rel="noreferrer nofollow" target="_blank">abstract base class</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">), </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , with one method, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . We need to add this to the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" rel="noreferrer nofollow" target="_blank">stub for </a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi" rel="noreferrer nofollow" target="_blank">os.py</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , as follows:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">class PathLike(</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-84113787329">Generic[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> @abstractmethod</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> def __fspath__(self) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> AnyStr: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">That’s really all there is to it (except for the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">sys.version_info</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> check, which I’ll leave out here since it doesn’t really work yet). Next we define </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.fspath()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which wraps this protocol. It’s slightly more complicated than just calling its argument’s </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> method, because it also handles strings and bytes. So here it is:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def fspath(path: PathLike[AnyStr]) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> AnyStr: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def fspath(path: AnyStr) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> AnyStr: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Easy enough! Next is update the definition of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . That’s easy too — in fact we only need to make it inherit from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , the rest is the same as the definition I gave above:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">class DirEntry(PathLike[AnyStr], Generic[AnyStr]):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> # Everything else unchanged!</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The only slightly complicated bit here is the extra base class </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Generic[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . This seems redundant, and in fact PEP 484 says we can leave it off, but mypy doesn’t support that yet, and it’s quite harmless — this just rubs into mypy’s face that this is a generic class of one type variable (the by-now famous </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ).</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Finally we need to make a similar change to the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3.4/pathlib.pyi" href="https://github.com/python/typeshed/blob/master/stdlib/3.4/pathlib.pyi" rel="noreferrer nofollow" target="_blank">stub for </a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3.4/pathlib.pyi" href="https://github.com/python/typeshed/blob/master/stdlib/3.4/pathlib.pyi" rel="noreferrer nofollow" target="_blank">pathlib.py</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . Again, all we need to do is to make </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PurePath</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> inherit from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , like so:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from os import PathLike</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">class PurePath(PathLike[str]):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-86168840759"> # Everything else unchanged!</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">However, here we don’t add </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Generic</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , because this is not a generic class! It inherits from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which is quite un-generic, since it’s </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>specialized</i></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> for just </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> .</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Note that we don’t actually have to define the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> method in these stubs — we’re not supposed to call them directly, and stubs don’t provide implementations, only interfaces.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Putting it all together, we see that it’s quite elegant:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">for a in os.scandir('.'):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> b = os.fspath(a)</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> # Here, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-459852399">the</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> typechecker will know that the type of b is str!</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The derivation that </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">b</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> has type </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is not too complicated: first, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.scandir('.')</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> has a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> argument, so it returns an iterator of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> objects parameterized with </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which we write as </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . Passing this </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> to </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.fspath()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> then takes the first of that function’s two overloads (the one with </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ), since it doesn’t match the second one ( </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> doesn’t inherit from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , because it’s neither a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> nor </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ). Further the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> type variable in </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is solved to stand for just </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , because </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry[str]</span><span class=""> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> inherits from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . This is the specialized version of what the code says: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">DirEntry[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> inherits from </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> .</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Okay, so maybe that last paragraph was intermediate or advanced. And maybe it could be expanded. Maybe I’ll write another blog about how type inference works, but there’s a lot on that topic, and other authors have probably already written better introductory material about generics (in other languages, though).</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Making things accept PathLike</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">There’s a bit of cleanup work that I’ve left out. PEP 519 says that many stdlib functions that currently take strings for pathnames will be modified to also accept </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . For example, here’s how the signatures for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> would change:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">def scandir() -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">def scandir(path: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-21653656371">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[AnyStr]]: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def scandir(path: PathLike[AnyStr]) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[AnyStr]]: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The first two entries are unchanged; I’ve just added a third overload. (Note that the alternative way of defining </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">scandir()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> would require more changes — an indication that this way is more natural.)</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">I also tried doing this with a union:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def scandir() </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">-</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[str]]: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">def scandir(path: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-21653656371">Union[AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283">, PathLike[AnyStr]]) -</span></code><code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z thread-53340393283"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">></span> Iterator[DirEntry[AnyStr]]: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">But I couldn’t get this to work, so the extra overload is probably the best we can do. Quite a few functions will require a similar treatment, sometimes introducing overloading where none exists today (but that shouldn’t hurt anything).</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">A note about </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">pathlib</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> : since it only deals with strings, its methods (the ones that PEP 519 says should be changed anyway) should use </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[str]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> rather than </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">PathLike[AnyStr]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> .</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Acknowledgments</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">(Thanks for comments on the draft to Stephen Turnbull, Koos Zevenhoven, Eth</span><span class="author-d-4z65zz66zl57z75zyiz66zfr2fz87zwz89znuiz90zz78zoz72zz87zhgh7z71zz88zz77zfz66zquz87zq3xz82zcz82zq5caz88z9">a</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">n Furman, and Brett Cannon.)</span></div>
Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com3tag:blogger.com,1999:blog-4195135246107166251.post-14686185153245976532016-05-17T09:53:00.002-07:002016-05-17T11:40:18.303-07:00The AnyStr type variable<div>
<h1 class="ace-copy-paste-skip-this-tag">
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The AnyStr type variable </span></h1>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">I was drafting a blog post on how to add type annotations for the new </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">__fspath__()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> protocol (</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0519/" href="https://www.python.org/dev/peps/pep-0519/" rel="noreferrer nofollow" target="_blank">PEP 519</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">) when I realized that I should write a separate post about </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . So here it is.</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">A simple function on strings</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Let’s write a function that surrounds a string in parentheses. We’ll put it in a file named </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">demo.py</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> :</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return '(' + s + ')'</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">It works, too:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">>>> from demo import parenthesize</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">>>> print(parenthesize('hola'))</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">(hola)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Of course, if you pass it something that’s not a string it will fail:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">>>> parenthesize(42)</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Traceback (most recent call last):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> File "demo.py", line 1, in <module></module></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> File "demo.py", line 2, in parenthesize</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">TypeError: Can't convert 'int' object to str implicitly</span></code></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Adding type annotations</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Using </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0484/" href="https://www.python.org/dev/peps/pep-0484/" rel="noreferrer nofollow" target="_blank">PEP 484</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> type annotations we can clarify our little function’s signature:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: str) -> str:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return '(' + s + ')'</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Nothing to it, right? Even if you’ve never heard of PEP 484 before you can guess what this means. (Note that PEP 484 also says that the runtime behavior is unchanged. The calls I showed above will still have exactly the same effect, including the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">TypeError</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> raised by </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">parenthesize(42)</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> .)</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Polymorphic functions</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Now suppose this is actually part of a networking app and we need to be able to parenthesize byte strings as well as text strings. Here’s how you’d implement that:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> if isinstance(s, str):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return '(' + s + ')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> elif isinstance(s, bytes):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return b'(' + s + b')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> else:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> raise TypeError(f"That's not a string, it's a {type(s)}") # See </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://www.python.org/dev/peps/pep-0498/" href="https://www.python.org/dev/peps/pep-0498/" rel="noreferrer nofollow" target="_blank">PEP 498</a></span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">With a fancy word we call that a polymorphic function. How do you write a signature for such a function? For the answer we have to dive a little deeper into PEP 484. It defines a nifty operator named </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Union</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> that lets us state that a type can be either this or that (or something else). In our case, it’s either </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> or </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , so we can write it like this:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import Union</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: Union[str, bytes]) -> Union[str, bytes]:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> if isinstance(s, str):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> # Etc.</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Now let’s write a little main program with a bug, to show off the type checker:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from demo import parenthesize</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">a = parenthesize('hello')</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">b = parenthesize(b'hola')</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">c = a + b ### bug here<-- bug="" span=""></--></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">print(c)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">When we try to run this, the two </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">parenthesize()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> calls work fine (yay polymorphism!) but we get a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">TypeError</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> on the last line:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">$ python3 main.py </span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Traceback (most recent call last):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> File "main.py", line 5, in <module></module></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> c = a + b ### bug here<-- bug="" span=""></--></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">TypeError: Can't convert 'bytes' object to str implicitly</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The reason should be pretty obvious: in Python 3 you can’t mix bytes and str objects. And when we type-check this program using </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://mypy-lang.org/" href="http://mypy-lang.org/" rel="noreferrer nofollow" target="_blank">mypy</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> we indeed get a type error:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">$ mypy main.py </span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">main.py:5: error: Unsupported operand types for + (likely involving Union)</span></code></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Debugging the bug</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">So let’s try a program without a bug:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from demo import parenthesize</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">a = parenthesize('hello')</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">b = parenthesize('hola')</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">c = a + b ### bug here<-- bug="" no="" span=""></--></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">print(c)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Run it and it works great:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">$ python3 main.py</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">(hello)(hola)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">So the type checker should be happy too, right?</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">$ mypy main.py</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">main.py:5: error: Unsupported operand types for + (likely involving Union)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Whoops! The same error. What happened? Of course, I set you up, so I can explain something about type checking.</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The trouble with </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z s"><s>tribbles</s></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> unions</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The type checker takes the signature at face value, so that when checking the call, it infers the type </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Union[str, bytes]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> for every call to </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">parenthesize()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , regardless of what the arguments are. This is because, for most functions of even modest complexity, a type checker doesn’t understand enough about what’s going on in the function body, so it just has to believe the types in the signature (even though in this particular case it would probably be easy enough to do better).</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">In our test program the types of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">a</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">b</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> are both inferred to be exactly what </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">parenthesize()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> claims to return, i.e., both variables have the type </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">Union[str, bytes]</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . The type checker then analyzes the expression </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">a + b</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , and for this i</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">t</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> discovers a problem: if </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">a</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is either str or bytes, and so is </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">b</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , then the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">+</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> operator may be invoked on any of these combinations of types: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str + str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str + bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes + str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , or </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes + bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . But only the first and the last are valid! In Python 3, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str + bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> or </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes + str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> are invalid operations.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Aside: Even in Python 2, those two are suspect: since while </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">'x' + u'y'</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> indeed works (returning </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">u'xy'</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ), other combinations will raise UnicodeDecodeError, e.g.:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">>>>'Franç' + u'ois'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Traceback (most recent call last):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> File "<stdin>", line 1, in <module></module></stdin></span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ruby" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ordinal not in range(128)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Anyway, the type checker doesn’t like this business, and it rejects operations on Unions where some combinations are invalid. What can we do instead?</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Function overloading</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">One option would be function overloading. PEP 484 defines a magical decorator, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">@overload</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , which lets us get around this problem. We could write something like this:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: str) -> str: ...</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">@overload</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: bytes) -> bytes: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">This tells the type checker that if the argument is a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , the return value is also a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> , and similarly for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . Unfortunately </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">@overload</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is only allowed in </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://mypy.readthedocs.io/en/latest/basics.html#library-stubs-and-the-typeshed-repo" href="http://mypy.readthedocs.io/en/latest/basics.html#library-stubs-and-the-typeshed-repo" rel="noreferrer nofollow" target="_blank">stub files</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, which are a kind of interface definition files that show a type checker the signatures of a module’s contents without giving the implementation.</span></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Type variables</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Fortunately there’s an even better way, using type variables. This is how it goes:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import TypeVar</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">S = TypeVar('S')</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: S) -> S:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> if isinstance(s, str):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return '(' + s + ')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> elif isinstance(s, bytes):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return b'(' + s + b')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> else:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> raise TypeError("That's not a string, dude! It's a %s" % type(s))</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Well… Almost. Our main.py program (unchanged from above) now gets a clean bill of health, but when we type-check this version we get errors on both </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">return</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> lines:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">demo.py: note: In function "parenthesize":</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">demo.py:7: error: Incompatible return value type: expected S`-1, got builtins.str</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">demo.py:9: error: Incompatible return value type: expected S`-1, got builtins.bytes</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">This is a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/mypy/issues/1539" href="https://github.com/python/mypy/issues/1539" rel="noreferrer nofollow" target="_blank">bit hard to fathom</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, but the fix is what I was leading up to anyway, so I’ll reveal it now:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import TypeVar</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">S = TypeVar('S', str, bytes)</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: S) -> S:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> if isinstance(s, str):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return '(' + s + ')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> elif isinstance(s, bytes):</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> return b'(' + s + b')'</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> else:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> raise TypeError("That's not a string, dude! It's a %s" % type(s))</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The only changed line is this one:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-bash" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">S = TypeVar('S', str, bytes)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">This notation is called a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="http://mypy.readthedocs.io/en/latest/generics.html#type-variables-with-value-restriction" href="http://mypy.readthedocs.io/en/latest/generics.html#type-variables-with-value-restriction" rel="noreferrer nofollow" target="_blank">type variable with value restriction</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . Yes, it’s mouthful; we sometimes also call it a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>constrained type variable</i></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">S</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is a type variable restricted to a set of types. It also has the advantage of telling the type checker that types other than </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> or </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> are not acceptable. Without that, a call like this would have been considered valid:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-ini" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">x = parenthesize(42)</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">because the original type variable (without the restrictions) doesn't tell mypy that this is a bad idea.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">In fact, this particular use case (a type variable constrained to str or bytes) is so commonly needed that it's predefined in the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">typing</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> module, and all we have to do is import it:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from typing import AnyStr</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1" spellcheck="false"><br /></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def parenthesize(s: AnyStr) -> AnyStr:</span></code></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> # Etc. -- trust me, it works!</span></code></div>
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Real-world use of AnyStr</span></h2>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">In fact, this is how many polymorphic functions in the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.path</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> modules are defined. For example, in the stub for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">os.py</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> we find definitions like </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi#L236" href="https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi#L236" rel="noreferrer nofollow" target="_blank">the following</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def link(src: AnyStr, link_name: AnyStr) -> None: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">and also </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z attrlink url"><a class="attrlink" data-target-href="https://github.com/python/typeshed/blob/master/stdlib/3/os/path.pyi#L57" href="https://github.com/python/typeshed/blob/master/stdlib/3/os/path.pyi#L57" rel="noreferrer nofollow" target="_blank">this</a></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">:</span></div>
<div>
<br /></div>
<div>
<code class="listtype-code listindent1 list-code1 lang-python" spellcheck="false"><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">def split(path: AnyStr) -> Tuple[AnyStr, AnyStr]: ...</span></code></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">These show us a bit more of the power of type variables: the signature for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">link()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> indicates that either both arguments must be </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> or both must be </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ; </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">split()</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> demonstrates that the type variable may also occur in more complex constructs: splitting a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> returns a tuple of two </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">str</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> objects, while splitting </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> returns a tuple of two </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">bytes</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> objects.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">That’s all I wanted to share about </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z inline-code">AnyStr</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> . Thanks for comments on the draft to Stephen Turnbull, Koos Zevenhoven, Ethan Furman, and Brett Cannon.</span></div>
<div>
<br /></div>
Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com5tag:blogger.com,1999:blog-4195135246107166251.post-2189453250114472282016-04-27T10:17:00.002-07:002020-12-03T10:22:37.937-08:00King's Day SpeechToday the Netherlands celebrates King's Day. To honor this tradition, the Dutch embassy in San Francisco invited me to give a "TED talk" to an audience of Dutch and American entrepreneurs. Here's the text I read to them. Part of it is the tl;dr of my autobiography; part of it is about the significance of programming languages; part of it is about Python's big idea. Leve de koning! (Long live the king!)<br />
<div>
<h2>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Python: a programming language created by a community</span></h2>
</div>
<div>
</div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Excuse my ramblings. I’ll get to a point eventually.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Let me introduce myself. I’m a nerd, a geek. I’m probably somewhere on the autism spectrum. </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">‘</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">m</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> also</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a late bloomer. I graduated from college when I was 26. I was 45 when I got married</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">’</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">m now 60 years old</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">with</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a 14 year old son.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Maybe I just have a hard time with decisions</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">: </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">’ve</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> lived in the US for over 20 years and I am still </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">a permanent resident</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I'm no Steve Jobs or Mark Zuckerberg</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. But</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">at age</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> 35 I created </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">a</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming language</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> that got a bit of a following. W</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">hat happened next was pretty amazing. But I'll get to that.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">At age 10 my parents gave me an educational electronics kit. The kit was made by Philips</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and it was amazing. At first I just followed the directions and everything worked; later I figured out how to design my own circuits. My prized possessions were the kit's three </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">(!) </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">transistors.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I took one of my first electronics models, a blinking light, to show and tell in 5th grade. It was a total dud </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">—</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> nobody </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">cared</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> or understood its importance. I think that's one of my earliest memories </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">of finding myself</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a geek</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">: </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">until then I had just been a quiet quick learner.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">In high school I developed my nerdiness further </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">—</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I hung out with a few other kids interested in electronics, and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">during</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> physics</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> class</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> we sat in the back of the class discussing NAND gates while the rest of the class was still figuring out Ohm's law.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Fortunately our physics teacher had figured</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>us</i></span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> out: he employed us to build a digital timer that </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">he</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> used to demonstrate the law of gravity to the rest of the class. It was a great project and showed us that our skills were useful. The other kids still thought we were weird</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">: </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">it was the seventies and many were into smoking pot and rebelling</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">; another group was already preparing for successful careers as doctors or lawyers or tech managers.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">B</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">ut they left me alone, I left them alone, and I graduated </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">as one of the best of my year</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">After high school I went to the </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">University of Amsterdam</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">:</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">I</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">t was close to home, and to a teen growing up in the Netherlands in the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">seventies</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">, Amsterdam was the only cool city</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. (Yes, the student protests of 1968 did touch me a bit.) </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Much to my </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">high school </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">physics teacher's </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">surprise and </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">disappointment</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I chose to major in math, not physics. But looking back I think </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">it didn’t matter</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">In the basement of t</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">he science building </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">was</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a mainframe </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">computer</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">, and it was love at first sight. Card punches! Line printers! Batch jobs! More to the point, I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">quickly</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> learn</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ed to</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> program</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> in languages</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> with names</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> like Algol, Fortran and Pascal. Mostly forgotten names, but highly influential at the time. Soon I was, again, sitting in the back of class</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> ignoring the lecture, correct</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ing</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> my </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">computer </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">program</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">s</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">And why was that?</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">In</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> that basement, around the mainframe, something amazing was happening. There was a loosely-knit group of students and staff with similar interests</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and we exchanged tricks of the trade. We shared subroutines and programs. We united in our alliances against the mainframe staff, especially in the endless cat-and-mouse games over disk space</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">(Disk space</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> was precious in a way </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">you</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> cannot </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">understand</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> today.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">)</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">But the most important lesson I learned was about sharing: while most</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> of the</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming tricks I learned there died with the mainframe era, the idea that software </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">needs</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> to be shared is stronger than ever</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. Today we call it</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> open source</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, and it’s a movement</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Hold that thought!</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">At the time</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">, my immediate knowledge of the tricks </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> the trade</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> seemed to matter most though.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> The </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">mainframe’s operating system</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> group employed a few part-time students, and when t</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">hey posted</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a vacancy</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> applied, and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> got </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">the job</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">It</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> was a life-changing event</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">!</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Suddenly I had unlimited access to the mainframe </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">— </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">no more fighting for space or terminals</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> — </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">plus access to the source code for its operating system, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> dozen</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">s of</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> colleagues </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">who showed</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> me how all that stuff worked.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I now had my dream job, programming all day, with real customers</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">: other programmers, </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">the users of the mainframe</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> stalled my studies and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> essentially dropped out of colleg</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">e, and I</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> would not have graduated if not for </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">my</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> enlightened manager and a professor who hadn't given up on me. They nudged me towards finishing some </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">classes</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and pulled some strings</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> eventually, with much delay, I did graduate.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> Yay!</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I immediately landed a new</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> dream</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> job that would not have been open to me without th</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">at</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> degree. I had never lost my interest in programming languages as an object of study</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I joined a team</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> building</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z i"><i>new</i></span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming languag</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">e — not something you see every day.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> designers hoped </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">their language </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">would take over the world, replacing Basic.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">It was the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">eighties</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> now, and</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> Basic was the language of choice for</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> a new generation of amateur programmers</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> coding</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> on</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> microcomputers like the Apple </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">II</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> the</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Commodore</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">64</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. Our team considered the Basic language a pest that the world should be rid of</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. The language </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">we were building</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">, ABC, would "stamp out Basic", according to our</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">motto.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Sadly, for a variety of reasons, our marketing (or perhaps our timing) sucked, and after four years</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> ABC was </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">abandoned</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> Since then</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I've spent </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">many</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> hours trying to understand why the project failed, despite its heart being so clearly in the right place. Apart from being somewhat over-engineered, my best </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">answer</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> is that ABC died because there was no internet in those days, and as a result there could not be a healthy feedback loop between the </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">makers</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> of the language and its users.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> ABC’s design was essentially a one-way street.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Just half a decade later, when I was picking through ABC’s ashes looking for ideas for my own language, that missing feedback loop was one of the things I decided to improve upon. “Release early, release often” became my motto (freely after the old Chicago Democrats’ encouragement, “vote early, vote often”). And the internet, small and slow as it was in 1990, made it possible.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Looking back 25 years, t</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">he Internet and the Open Source movement (a.k.a. Free Software)</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> really did</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> change everything. Plus something called Moore's Law, which makes </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">computers</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> faster every year. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Together, these</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> have entirely changed the interaction between the makers and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">u</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">s</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ers</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> of computer software. It is my belief that t</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">hese developments</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> (and how I managed to make good use of them) have contributed more to the success of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">“my”</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming language than my programming skills and experience</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, no matter how awesome.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">It also didn't hurt that I named </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">my language</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Python. This was a bit of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">unwitting</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> marketing genius on my part. I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">meant</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> to honor the irreverent comedic genius of Monty Python's Flying Circus, and back in 1990 I didn't think I had much to lose. Nowadays, I'm sure "brand research" firms would be happy to charge you a very large fee to </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">tell you</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> exactly what complex of associations this name tickles in the subconscious of the typical customer. But I was just being flippant.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">have promised the ambassador not to</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> bore you with</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> a technical discussion of the merits of different</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming language</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">s</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. But I would like to say a few things about what programming languages mean to the people who use them </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">—</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programmers. Typically when you ask a programmer to explain to a lay person what a programming language is, they will say that it is how you tell a computer what to do. But if that was all, why would they be so passionate about </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">programming </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">languages when they talk among themselves?</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">In reality, programming languages are how programmers express and communicate </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z i"><i>ideas</i></span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">— </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">and the audience for those ideas is other programmers, not computers. The reason: the computer can take care of itsel</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">f,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> but programmers are always working with other programmers, and</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> poorly</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">communicated</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ideas can</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> cause expensive </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">flops</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. In fact, ideas expressed in a programming language also often reach the end users of the program </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">—</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> people who will never read or even know about the program, but who nevertheless are affected by it.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">Think of the incredible success of companies like Google or Facebook. At the core of t</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">hese</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">are</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> ideas </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">—</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> ideas about what computers can do for people. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">To</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> be effective, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">an</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> idea must be express</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">ed</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> as</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> a</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> computer program, using</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> a</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> programming language.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> The language that is best to express an idea will give the team using that language a key advantage, because it gives the team members — people! — clarity about that idea.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> The ideas underlying Google and Facebook couldn't be more different, and indeed these companies' favorite programming languages are at opposite ends of the spectrum of programming language design.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> And that’s exactly my point.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">True story: The first version of Google was written in Python. The reason: Python was the right language to express the original </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>ideas</i></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> that Larry Page and Sergey Brin had about how to index the web and organize search results. And they could run their ideas on a computer, too!</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">So</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, in 1990, long before Google and Facebook,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">made</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">my own </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">programming language</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">named</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> it Python.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">But what is the </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z i"><i>idea</i></span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> of Python? Why is it so successful? How does Python distinguish itself from other programming languages?</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>(Why are you all staring at me like that? :-)</i></span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">I have</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> many answers, some quite technical, some </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">from my specific skills and experience at the time, some just about being in the right place at the right time</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">. But I believe the most important idea is that Python is developed on the Internet, entirely in the open, by a community of volunteers (but not amateurs!) who feel </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">passion and</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> ownership.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">And </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>that</i></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> is what that group of geeks in the basement of the science building was all about.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z i"><i>Surprise:</i></span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> Like any good inspirational speech, the point of this talk is about happiness!</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">am</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">happiest</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> when I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">feel that</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> I'm part of s</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">uch a</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> community</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. I’m lucky that I can feel it</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">in my day job too. (</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I'm</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">a </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">principal</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> engineer at Dropbox</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">.)</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> If I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">can't feel it, </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">I don't feel alive. And so it is for the other community members. </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">The</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> feeling is contagious, and there are members of our community all over the world.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">The </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Python user</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> community </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">is formed of millions of </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">people who consciously use Python</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> and love using it</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">. There are</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> active members organizing </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">Python </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">conferences</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> — affectionately known as PyCons —</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> in faraway places like Namibia, Ira</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">n,</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Ira</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">q, even Ohio!</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">My favorite story: </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">A</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> year ago I spent 20 minutes on a video conference</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> call</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> with a classroom full of faculty and staff at Babylon University in southern Iraq</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">, answering questions about Python. T</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">hanks to the efforts of </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">the</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> audacious woman</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> who organized this event in a war-ridden country</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">, </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">students</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">at Babylon University </span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">are now being taught</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> introductory programming classes</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> using Python. I still tear up when I think about the power of that experience.</span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z"> In my wildest dreams I never expected I’d touch lives so far away and so different from my own.</span></div>
<div>
<br /></div>
<div>
<span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z">And on that note I'd like to leave you: a programming language created by a community </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">fosters happiness in its users around the world.</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Next year I </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">may</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> go to </span><span class="author-d-16z86ztz122z98z81zz82zz85zunv3z82zpqfnlaklz69zehdlvnz73zz65zz81zz79zz73zpz76z22z66ztsz89zz122zz73zz122zfz83z">PyCon</span><span class="author-d-z89zz72zz79zvhpz67zz83z9z66zz78zxz122z1xz74zu4z83z4myz73zkiz71zdz77zz71zz65zz79z4iz79ziosz75zz85zreqz69z"> Cuba!</span></div>
Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com7tag:blogger.com,1999:blog-4195135246107166251.post-31676554598137723492013-10-28T13:56:00.001-07:002013-10-28T13:56:16.732-07:00Book review: Introduction to Computer Science Using Python (by Charles Dierbach)After much back and forth I received a nice new Python book in the mail. The book's full title is "<a href="http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Dstripbooks&field-keywords=Introduction-Computer-Science-Using-Python+dierbach&rh=n%3A283155%2Ck%3AIntroduction-Computer-Science-Using-Python+dierbach">Introduction to Computer Science Using Python: A Computational Problem-Solving Focus</a>", and its author is a very experienced educator, Charles Dierbach.<br />
<br />
This is not your average Python book -- it is a college text intended for first-semester CS courses that happens to use Python. As such, in assumes absolutely no previous programming experience, and it looks like any previous computer experience is optional. Not only that, but the book starts with a step-by-step introduction to the art of computational problem solving. This is an idea that goes well beyond hacking together a website!<br />
<br />
The book is incredibly thorough: there are exercises throughout the text (not just at the end of each chapter), and it includes a plethora of examples, screenshots, tables, charts, diagrams, and photos. (Yes, my picture is in there -- so are Alan Turing, JFK, and K&R. :-)<br />
<br />
The author is not afraid of taking a stance; for example, he omits the 'break' and 'continue' statement because they do not fit within the paradigm of structured programming. This actually fits with the general goal of the book, which is to give an overview of many areas of computer science without getting too deep into the minutiae of any topic. I love the final chapter, which is an overview of the history of computing, starting with Charles Babbage and Ada Lovelace.<br />
<br />
At the same time, the book gives plenty of useful practical information, such as instructions for using IDLE and an extensive explanation of turtle graphics, culminating in a horse race simulation. (The author's Baltimore roots seem to show through here. :-)<br />
<br />
All in all, I think this book is a great text for anyone teaching CS1 or interested in familiarizing themselves with computer science through serious self-study.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com7tag:blogger.com,1999:blog-4195135246107166251.post-13653330640343969882013-10-24T19:19:00.002-07:002013-11-22T15:31:23.553-08:00Letter to a young programmerDear <insert here="" name="">(insert name here),</insert><br />
<br />
I heard you enjoy a certain programming language named Python. Programming is a wonderful activity. I am a little jealous that you have access to computers at your age; when I grew up I didn't even know what a computer was! I was an electronics hobbyist though, and my big dream was to build my own electronic calculator from discrete components. I never did do that, but I did build several digital clocks, and it was amazing to build something that complex and see it work. I hope you dream big too -- programmers <span class="il">can </span>make computers (and robots!) do amazing things, and this is a great time to become a programmer. Just imagine how much faster computers will be in five or ten years, and what you will be able to do with your skills then!<br />
<br />
--Guido van Rossum (inventor of Python) Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com18tag:blogger.com,1999:blog-4195135246107166251.post-4668752163530648162011-08-25T09:59:00.000-07:002011-08-29T07:57:50.355-07:00Compare-And-Set in MemcacheWith the most recent release (1.5.3, last week) App Engine's Python API for Memcache has added a new feature, <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_cas">Compare-And-Set</a>. This feature (with a <a href="http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/memcache/MemcacheService.html#putIfUntouched%28java.lang.Object,%20com.google.appengine.api.memcache.MemcacheService.IdentifiableValue,%20java.lang.Object%29">different API</a>) was already available in Java; it has also been available in the non-App-Engine <a href="http://www.tummy.com/Community/software/python-memcached/">pure-Python memcache client</a>. In fact, I designed the App Engine Python API for this feature to be compatible with the latter, since most of the rest of the App Engine Python API also strives to be at least a superset of that package.
<br />
<br />But what is it? There seems to be little information on how to use Compare-And-Set with memcache. It is also sometimes (incorrectly) referred to as Compare-And-Swap -- incorrect, because the cas() operation does not actually "swap" anything. The first response when we closed <a href="http://code.google.com/p/googleappengine/issues/detail?id=2139">the bug requesting this feature</a> was <span style="font-style: italic;">"Some examples of usage are appreciated."</span> So here goes.
<br />
<br />The basic use case for Compare-And-Set is when multiple requests that are being handled concurrently need to update the same memcache key in an atomic fashion. Let's assume you are managing a counter in memcache. (Actually, you could use the <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_incr">incr() and decr(</a>) operations to update 64-bit integer counters atomically, but just for argument's sake assume you cannot use those -- there are other data types for which the memcache service does not have built-in support.)
<br />
<br />The naive code to update a counter would be something like this:
<br />
<br />def init_counter(key):
<br />. memcache.set(key, 0)
<br />
<br />def bump_counter(key):
<br />. counter = memcache.get(key)
<br />. assert counter is not None, 'Uninitialized counter'
<br />. memcache.set(key, counter+1)
<br />
<br />(Aside: I don't want to have to think about how to get blogger to properly format Python code. I really don't. So just bear with the dots I use for indentation. Okay? Comments pointing me to solutions will be DELETED.)
<br />
<br />(Aside 2: The assert is kind of naive; in practice you'll have to somehow deal with counter initialization. You also should implement a backup for your counter using the App Engine datastore, so that it can survive eviction by the memcache service. However interesting these details are on their own, I leave them for another time.)
<br />
<br />Hopefully you can spot the problem in this version of bump_counter(): if two requests execute concurrently (on different <a href="http://code.google.com/appengine/docs/adminconsole/instances.html">instances</a> of the same app), the sequence of operations might be as follows, labeling the two requests as A and B:
<br />
<br />A: counter = memcache.get(key) # Reads 42
<br />B: counter = memcache.get(key) # Reads 42
<br />A: memcache.set(key, counter+1) # Writes 43
<br />B: memcache.set(key, counter+1) # Writes 43
<br />
<br />So even though two requests were executed, the counter only gets incremented by one. This is called a <span style="font-style: italic;">race condition</span>. Various interlacings of these lines can have the same effect; a race condition occurs whenever B reads the counter before A has written it (or vice versa).
<br />
<br />You could try to guard against this by reading the counter value back and checking that it was incremented by one; however this solution still has a race condition (see if you can figure it out for yourself). There are other solutions possible involving a separate "lock" variable, managed using the <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_add">add()</a> and <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_delete">delete()</a> operations. However these in general require more server roundtrips, and it is pretty hard to manufacture a decent lock out of the basic memcache operations (try for yourself -- think about what would happen if your request was somehow aborted after acquiring the lock, without having a chance of releasing it).
<br />
<br />Using the Compare-And-Set operation, writing a reliable bump_counter() function is a cinch:
<br />
<br />def bump_counter(key):
<br />. <span style="font-weight: bold;">client = memcache.Client()</span>
<br />. <span style="font-weight: bold;">while True: # Retry loop</span>
<br />. . counter = <span style="font-weight: bold;">client.gets</span>(key)
<br />. . assert counter is not None, 'Uninitialized counter'
<br />. . <span style="font-weight: bold;">if</span> <span style="font-weight: bold;">client.cas</span>(key, counter+1):
<br />. . . <span style="font-weight: bold;">break</span>
<br />
<br />I've highlighted the changes from the previous version. There are several essential differences:
<br /><ul><li>The use of a memcache <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html">Client object</a> instead of memcache <a href="http://code.google.com/appengine/docs/python/memcache/functions.html">functions</a></li><li>The use of a retry loop</li><li>The use of <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_gets">gets()</a> and <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_cas">cas()</a> instead of get() and set()<span style="font-style: italic;"></span></li></ul>The Client object is required because the gets() operation actually squirrels away some hidden information that is used by the subsequent cas() operation. Because the memcache functions are <span style="font-style: italic;">stateless</span> (meaning they don't alter any global values), these operations are only available as methods on the Client object, not as functions in the memcache module. (Apart from these two, the methods on the Client object are exactly the same as the functions in the module, as you can tell by comparing the documentation.)
<br />
<br />The retry loop is necessary because this code doesn't actually avoid race conditions -- it just detects them! The memcache service guarantees that when used in the pattern shown here (i.e. using gets() instead of get() and cas() instead of set()), if two (or more) different client instances happen to be involved a race condition like I showed earlier, only the first one to execute the cas() operation will succeed (return True), while the second one (and later ones) will fail (return False). Let's spell out the events that happen when a race condition occurs:
<br />
<br />A: counter = memcache.gets(key) # Reads 42
<br />B: counter = memcache.gets(key) # Reads 42
<br />A: memcache.cas(key, counter+1) # Writes 43, returns True
<br />B: memcache.cas(key, counter+1) # Returns False
<br />B: counter = memcache.gets(key) # Reads 43
<br />B: memcache.cas(key, counter+1) # Writes 44, returns True
<br />
<br />Another refinement I've left out here for brevity is to set a limit on the number of retries, to avoid an infinite loop in worst-case scenarios where there is a lot of contention for the same counter (meaning more requests are trying to update the counter than the memcache service can process in real time). You can figure out how to code this for yourself. (<span style="font-weight: bold;">UPDATE:</span> see comments #1 and #2 for a note about busy-waiting.)
<br />
<br />Now let me explain roughly how this actually works. For some people that helps understanding how to use it: this is often the case for me when I am trying to understand some new concept.
<br />
<br />The gets() operation internally receives <span style="font-style: italic;">two</span> values from the memcache service: the value stored for the key (in our example the counter value), and a <span style="font-style: italic;">timestamp</span> (also known as the <span style="font-style: italic;">cas_id</span>). The timestamp is an opaque number; only the memcache service knows what it means. The important thing is that each time the value associated with a memcache key is updated, the associated timestamp is changed. The gets() operation stores this timestamp in a Python dict on the Client object, using the key passed to gets() as the dict key.
<br />
<br />The cas() operation internally adds the timestamp to the request it sends to the memcache service. The service then compares the timestamp received with a cas() operation to the timestamp currently associated with the key. If they match, it updates the value and the timestamp, and returns success. If they don't match, it leaves the value and timestamp alone, and returns failure. (By the way, it does not send the new timestamp back with a successful response. The only way to retrieve the timestamp is to call gets().)
<br />
<br />Of course, there's one more important ingredient: the App Engine memcache service itself behaves atomically. That is, when two concurrent requests (for the same app id) use memcache, they will go to the same memcache service instance (for historic reasons called a <span style="font-style: italic;">shard</span>), and the memcache service has enough internal locking so that concurrent requests for the same key are properly serialized. In particular this means that two cas() requests for the same key do not actually run in parallel -- the service handles the first request that came in until completion (i.e., updating the value and timestamp) before it starts handling the second request.
<br />
<br />And that's Compare-And-Set in a nutshell. If you have questions please don't hesitate to ask!
<br />
<br />(<span style="font-weight: bold;">UPDATE:</span> The memcache API defines batch versions of most of its APIs. For example, to get multiple keys in a single call, there is get_multi(); to set multiple keys, there is set_multi(). Corresponding to cas(), there is <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_cas_multi">cas_multi()</a>. But there is no gets_multi(): instead, you can use <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_get_multi">get_multi(keys, for_cas=True)</a>. Finally, there's <a href="http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_cas_reset">cas_reset()</a>, which clears the dict used to store timestamps. But I haven't figured out what to do with it yet. :-)
<br />Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com17tag:blogger.com,1999:blog-4195135246107166251.post-12978155849599188322011-07-25T11:17:00.000-07:002011-07-25T13:21:43.049-07:00Before PythonThis morning I had a chat with the students at Google's <a href="http://www.google.com/jobs/cape/">CAPE</a> program. Since I wrote up what I wanted to say I figured I might as well blog it here. Warning: this is pretty unedited (or else it would never be published :-). I'm posting it in my "personal" blog instead of the "Python history" blog because it mostly touches on my career <i>before</i> Python. Here goes.<div><br /></div><div><div>Have you ever written a computer program? Using which language?</div><div><ul><li>HTML</li><li>Javascript</li><li>Java</li><li>Python</li><li>C++</li><li>C</li><li>Other - which?</li></ul><div>[It turned out the students had used a mixture of <a href="http://scratch.mit.edu/">Scratch</a>, <a href="http://appinventor.googlelabs.com/about/">App Inventor</a>, and <a href="http://processing.org/">Processing</a>. A few students had also used Python or Java.]</div></div><div><br /></div><div>Have you ever invented a programming language? :-)</div><div><br /></div><div>If you have programmed, you know some of the problems with programming languages. Have you ever thought about why programming isn't easier? Would it help if you could just talk to your computer? Have you tried speech recognition software? I have. It doesn't work very well yet. :-)</div><div><br /></div><div>How do you think programmers will write software 10 years from now? Or 30? 50?</div><div><br /></div><div>Do you know how programmers worked 30 years ago?</div><div><br /></div><div>I do.</div><div><br /></div><div>I was born in Holland in 1956. Things were different.</div><div><br /></div><div>I didn't know what a computer was until I was 18. However, I tinkered with electronics. I built a digital clock. My dream was to build my own calculator.</div><div><br /></div><div>Then I went to <a href="http://www.english.uva.nl/">university</a> in Amsterdam to study mathematics and they had a <a href="http://60bits.net/">computer</a> that was free for students to use! (Not unlimited though. We were allowed to use something like one second of CPU time per day. :-)</div><div><br /></div><div>I had to learn how to use <a href="http://en.wikipedia.org/wiki/Punched_card">punch cards</a>. There were machines to create them that had a keyboard. The machines were as <a href="http://www.columbia.edu/cu/computinghistory/029.html">big as a desk</a> and made a terrible noise when you hit a key: a small hole was punched in the card with a huge force and great precision. If you made a mistake you had to start over.</div><div><br /></div><div>I didn't get to see the actual computer for several more years. What we had in the basement of the <a href="http://staff.science.uva.nl/%7Epeter/portretpeter.jpg">math department</a> was just an end point for a network that ran across the city. There were card readers and line printers and operators who controlled them. But the actual computer was elsewhere.</div><div><br /></div><div>It was a huge, busy place, where programmers got together and discussed their problems, and I loved to hang out there. In fact, I loved it so much I nearly dropped out of university. But eventually I graduated.</div><div><br /></div><div>Aside: Punch cards weren't invented for computers; they were invented for sorting census data and the like before WW2. [UPDATE: actually <a href="http://en.wikipedia.org/wiki/Punched_card">much earlier</a>, though the <a href="http://en.wikipedia.org/wiki/Punched_card#IBM_80_column_punched_card_format">IBM 80-column format</a> I used did originate in 1928.] There were large mechanical machines for sorting stacks of cards. But punch cards are the reason that some software still limits you (or just defaults) to 80 characters per line.</div><div><br /></div><div>My first program was a kind of "hello world" program written in <a href="http://en.wikipedia.org/wiki/ALGOL_60">Algol-60</a>. That language was only popular in Europe, I believe. After another student gave me a few hints I learned the rest of the language straight from the official definition of the language, the "<a href="http://www.masswerk.at/algol60/report.htm">Revised Report on the Algorithmic Language Algol-60</a>." That was not an easy report to read! The language was a bit cumbersome, but I didn't mind, I learned the basics of programming anyway: variables, expressions, functions, input/output.</div><div><br /></div><div>Then a professor mentioned that there was a new programming language named Pascal. There was a Pascal compiler on our mainframe so I decided to learn it. I borrowed the <a href="http://www.anvari.org/fortune/Miscellaneous_Collections/228172_silver-book-n-jensen-and-wirths-infamous-pascal-user-manual-and-repo.html">book</a> on Pascal from the departmental library (there was only one book, and only one copy, and I couldn't afford my own). After skimming it, I decided that the only thing I really needed were the "railroad diagrams" at the end of the book that summarized the language's syntax. I made photocopies of those and returned the book to the library.</div><div><br /></div><div>Aside: Pascal really had only one new feature compared to Algol-60, pointers. These baffled me for the longest time. Eventually I learned assembly programming, which explained the memory model of a computer for the first time. I realized that a pointer was just an address. Then I finally understood them.</div><div><br /></div><div>I guess this is how I got interested in programming languages. I learned the other languages of the day along the way: Fortran, Lisp, Basic, Cobol. With all this knowledge of programming, I managed to get a plum part-time job at the data center maintaining the mainframe's operating system. It was the most coveted job among programmers. It gave me access to unlimited computer time, the fastest terminals (still 80 x 24 though :-), and most important, a stimulating environment where I got to learn from other programmers. I also got access to a Unix system, learned C and shell programming, and at some point we had an Apple II (mostly remembered for hours of playing space invaders). I even got to implement a new (but very crummy) programming language!</div><div><br /></div><div>All this time, programming was one of the most fun things in my life. I thought of ideas for new programs to write all the time. But interestingly, I wasn't very interested in using computers for practical stuff! Nor even to solve mathematical puzzles (except that I invented a clever way of programming <a href="http://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Conway's Game of Life</a> that came from my understanding of using logic gates to build a binary addition circuit).</div><div><br /></div><div>What I liked most though was write programs to make the life of programmers better. One of my early creations was a text editor that was better than the system's standard text editor (which wasn't very hard :-). I also wrote an archive program that helped conserve disk space; it was so popular and useful that the data center offered it to all its customers. I liked sharing programs, and my own principles for sharing were very similar to what later would become Open Source (except I didn't care about licenses -- still don't :-).</div><div><br /></div><div>As a term project I wrote a static analyzer for Pascal programs with another student. Looking back I think it was a horrible program, but our professor thought it was brilliant and we both got an A+. That's where I learned about parsers and such, and that you can do more with a parser than write a compiler.</div><div><br /></div><div>I combined pleasure with a good cause when I helped out a small left-wing political party in Holland automate their membership database. This was until then maintained by hand as a collection of metal plates plates into which letters were stamped using an antiquated machine not unlike a steam hammer :-). In the end the project was not a great success, but my contributions (including an emulation of Unix's venerable "<a href="http://en.wikipedia.org/wiki/Ed_%28text_editor%29">ed</a>" editor program written in Cobol) piqued the attention of another volunteer, whose day job was as computer science researcher at the Mathematical Center. (Now <a href="http://cwi.nl/">CWI</a>.)</div><div><br /></div><div>This was <a href="http://en.wikipedia.org/wiki/Lambert_Meertens">Lambert Meertens</a>. It so happened that he was designing his own programming language, named B (later <a href="http://homepages.cwi.nl/%7Esteven/abc/">ABC</a>), and when I graduated he offered me a job on his team of programmers who were implementing an interpreter for the language (what we would now call a virtual machine).</div><div><br /></div><div>The rest I have <a href="http://python-history.blogspot.com/2009/01/personal-history-part-1-cwi.html">written up earlier</a> in my Python history blog.</div></div>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com23tag:blogger.com,1999:blog-4195135246107166251.post-92169309501650130302011-06-03T09:17:00.001-07:002011-07-08T16:34:22.895-07:00The depth and breadth of PythonAs of late I'm noticing a trend: I'm spending more time having in-person in-depth conversations, and less time coding. While I regret the latter, I really enjoy the former. Certainly more than weekly meetings, code reviews, or bikeshedding email threads. (I'm not all that excited about blogging either, as you may have guessed; but some things just don't fit in 140 characters.)<br /><br />Two conversations with visitors I particularly enjoyed this week were both with very happy Python users, and yet they couldn't be more different. This to me is a confirmation of Python's enduring depth and breadth: it is as far away of a one-trick language as you can imagine.<br /><br />My first visitor was <a href="http://www.cs.sunysb.edu/%7Eliu/">Annie Liu</a>, a professor of computer science (with a tendency to theory :-) at Stony Brook University in New York State. During an animated conversation that lasted nearly three hours (and still she had more to say :-) she explained to me the gist of her research, which appears to be writing small Python programs that implement fundamental algorithms using set comprehensions, and then optimizing the heck out of it using an automated approach she summarized as the three I's: Iterate, incrementalize, and implement. While her academic colleagues laugh at her for her choice of such a non-theoretical language like Python, her students love it, and she seems to be having the last laugh, obtaining publication-worthy results that don't require advanced LaTeX skills, nor writing in a dead language like SETL (of which she is also a great fan, and which, via ABC, had some influence on Python -- see also below).<br /><br />Annie told me an amusing anecdote about an inscrutable security standard produced by NiST a decade ago, with a fifty-page specification written in Z. She took a 12-page portion of it and translated it into a 120-line Python program, which was much more readable than the original, and in the process she uncovered some bugs in the spec!<br /><br />Another anecdote she recounted had reached me before, but somehow I had forgotten about it until she reminded me. It concerns the origins of Python's use of indentation. The anecdote takes place long before Python was created. At an IFIP working group meeting in a hotel, one night the delegates could not agree about the best delimiters to use for code blocks. On the table were the venerable BEGIN ... END, the newcomers { ... }, and some oddities like IF ... FI and indentation. In desperation someone said the decision had to be made by a non-programmer. The only person available was apparently Robert Dewar's wife, who in those days traveled with her husband to these events. Despite the late hour, she was called down from her hotel room and asked for her independent judgement. Immediately she decided that structuring by pure indentation was the winner. Now, I've probably got all the details wrong here, but apparently Lambert Meertens was present, who went on to design Python's predecessor, ABC, though at the time he called it <span style="font-style: italic;">B</span> (the italics meant that <span style="font-style: italic;">B</span> was not the name of the language, but the name of the variable containing the name of the language). I checked my personal archives, and the first time I heard this was from Prof. Paul Hilfinger at Berkeley, who recounted a similar story. In his version, it was just Lambert Meertens and Robert Dewar, and Robert Dewar's wife chose indentation because she wanted to go to bed. Either way it is a charming and powerful story. <span style="font-weight: bold;">(UPDATE: indeed <a href="http://python-history.blogspot.com/2011/07/karin-dewar-indentation-and-colon.html">the real story</a> was quite different.)</span><br /><br />Of course Annie had some requests as well. I'll probably go over these in more detail on python-ideas, but here's a quick rundown (of what I could remember):<br /><ul><li>Quantifiers. She is really longing for the "<a href="http://homepages.cwi.nl/%7Esteven/abc/qr.html#TESTS">SOME x IN xs HAS pred</a>" notation from ABC (and its sibling "EACH x IN xs HAS pred"), which superficially resemble Python's any() and all() functions, but have the added semantics of making x available in the scope executed when the test succeeds (or fails, in the case of EACH -- then x represents a counterexample).</li><li>Type declarations. (Though I think she would be happy with Python 3 function annotations, possibly augmented with the attribute declarations seen in e.g. Django and App Engine's model classes.)</li><li>Pattern matching, a la Erlang. I have been eying these myself from time to time; it is hard to find a syntax that really shines, but it seems to be a useful feature.</li><li>Something she calls labels or yield points. It seems somewhat similar to yield statements in generators, but not quite.</li><li>She has only recently begun to look at distributed algorithms (she had some Leslie Lamport anecdotes as well) and might prefer sets to be immutable after all. Though that isn't so clear; her work so far has actually benefited from mutating sets to maintain some algorithmic invariant. (The "incrementalize" of the three I's actually refers to a form of "differentiation" of expressions that produce a new set for each input.)</li></ul>The contrast with my visitor the next day couldn't be greater. Through a former colleague I got an introduction to Drew Houston, co-founder and CEO of the vastly successful start-up company <a href="http://www.dropbox.com/">Dropbox</a>. Dropbox currently has 25 million users, stores petabytes of data on Amazon S3, is profitable, and is not for sale. Drew is an easygoing MIT graduate who is equally comfortable discussing custom memory allocators, the world of venture capitalism, and how to keep engineers happy; he likes hard problems and winning.<br /><br />Python plays an important role in Dropbox's success: the Dropbox client, which runs on Windows, Mac and Linux (!), is written in Python. This is key to the portability: everything except the UI is cross-platform. (The UI uses a Python-ObjC bridge on Mac, and wxPython on the other platforms.) Performance has never been a problem -- understanding that a small number of critical pieces were written in C, including a custom memory allocator used for a certain type of objects whose pattern of allocation involves allocating 100,000s of them and then releasing all but a few. Before you jump in to open up the Dropbox distro and learn all about how it works, beware that the source code is not included and the bytecode is obfuscated. Drew's no fool. And he laughs at the poor competitors who are using Java.<br /><br />Next Monday I'm having lunch with another high-tech enterpreneur, a Y-combinator start-up founder using (and contributing to) App Engine. Maybe I should just cancel all weekly meetings and sign off from all mailing lists and focus on two things: meeting Python users and coding. That's the life!Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com38tag:blogger.com,1999:blog-4195135246107166251.post-2555809263743628302011-01-24T10:07:00.000-08:002011-01-24T10:40:22.262-08:00Asynchronous RPC in App Engine TodayWhile I was laying the groundwork for a <a href="http://neopythonic.blogspot.com/2011/01/new-app-engine-datastore-api.html">new datastore client library</a> with support for asynchronous requests, I added some low-level support for asynchronous RPCs that you can use today. The only App Engine API with documented support for asynchronous RPCs is <a href="http://code.google.com/appengine/docs/python/urlfetch/asynchronousrequests.html">urlfetch</a>, and it happens to be quite useful with that.<br /><br />Suppose you want to fetch some data from a remote service. The remote service has two instances, both of which are slightly flaky. What you want to do is send off requests to both servers simultaneous (this is the easy part) and then <span style="font-style: italic;">wait for the first one to give you a result</span>. The latter uses the new API that I'm about to describe here.<br /><pre><br />from google.appengine.api import urlfetch, apiproxy_stub_map<br /><br />urls = ['http://service1.com', 'http://service2.com'] # Etc.<br /><br />rpcs = []<br />for url in urls:<br /> rpc = urlfetch.create_rpc(deadline=1.0)<br /> urlfetch.make_fetch_call(rpc, url)<br /> rpcs.append(rpc)<br /><br />rpc = apiproxy_stub_map.UserRPC.wait_any(rpcs)<br /># Now rpc is the first rpc that returned a result. Have at it!<br /></pre><br />That's all! If you're interested in learning more about this handy class method, just check out its <a href="http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/apiproxy_stub_map.py#555">docstring in the App Engine SDK</a>. Note that technically you should loop until it doesn't return None.<br /><br />You can also repeatedly call wait_any() to get subsequent result. Make sure to remove the rpc it returns (if any) from the list, since otherwise it will return the same rpc over and over again: the specification of wait_any() says it returns the first rpc in the given list that completes, regardless of whether you have seen it before.<br /><br />Also note that there currently is no way to cancel the other RPCs, which is why I passed a low deadline to the<span style="font-style: italic;"></span> create_rpc() call. The problem is that even if you completely ignore the other RPCs, the App Engine runtime still waits for them to finish or timeout.<br /><br />Finally, there is also a similar class method UserRPC.wait_all(), which waits until <span style="font-style: italic;">all</span> RPCs in the list you pass it are complete. (It doesn't return anything.)<br /><br />PS. Don't look too closely at the implementation of these methods. It may change as we think of a better way to do it. But we're committed to the API.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com8tag:blogger.com,1999:blog-4195135246107166251.post-44933819857760305462011-01-07T12:20:00.000-08:002011-01-08T08:54:07.768-08:00A new App Engine datastore APIThis post is primarily intended for App Engine users (and of those, only Python users :-).<br /><br />Over the past months I've been working on a new design for the Python datastore API, under the code name Datastore Plus. The new design is very ambitious, and changes a lot of things:<br /><ul><li>New, cleaner implementations of Key, Model, Property and Query classes</li><li>High-level asynchronous API using Python generators as coroutines (PEP 342)</li></ul>The design is meant to eventually replace the existing db package in the App Engine runtime library, but for now, it is just an open source project which you have to download and copy into your application.<br /><br />I am not at all finished with this design, but I believe in listening to users, so I am making a preliminary version of the new API available for review. Please send me your thoughts, either in this blog, or via private mail to guido (at) google.com. Note that the implementation works, but I cannot guarantee that it won't change.<br /><br />Documentation is here: <a href="http://goo.gl/D6Onw">http://goo.gl/D6Onw</a><br /><br />The project to check out is here: <a href="http://goo.gl/GapXI">http://goo.gl/GapXI</a><br /><br />You must use Mercurial to check out the project, but it's fine to check it out anonymously -- I don't require anybody to log in to look or comment. (If using Mercurial is too much of a burden, there's also a zipfile on the site, but I don't plan to update it frequently.)<br /><br />I'm interested in receiving any kind of feedback at all. It would help me if you could clarify whether your feedback is about an issue with the documentation, an issue with the implementation, or an issue with the API design -- though I realize you can't always tell the difference. :-)<br /><br />(You can also comment on the thread in the google-appengine-python group here: <a href="https://groups.google.com/group/google-appengine-python/browse_thread/thread/454cb81d49e759f2">https://groups.google.com/group/google-appengine-python/browse_thread/thread/454cb81d49e759f2</a>.)Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com9tag:blogger.com,1999:blog-4195135246107166251.post-6773480855774313322009-12-11T13:45:00.001-08:002010-08-26T11:40:37.115-07:00While-you-type SearchingHere's an idea that is just begging to be implemented as a Firefox extension.<div><br /></div><div>You know how there's a while-you-type spell checker that's always on when you are editing text in a multi-line text box? There should be a feature that takes the last few words you're typing (or the entire current paragraph, or whatever works best), does a Google search, and presents snippets for the top few results in an unobtrusive pop-up window.<div><br /></div><div>Sure, maybe you're thinking "It looks like you're writing a letter. Do you want me to write it for you?" (the Microsoft paperclip). But using web search instead of a fixed set of patterns could actually make this useful. Imagine the number of messages to customer support forums that will never have to be sent because this feature pops up the answer the user was looking for. And so on.</div><div><br /></div><div>You might also think, "this already exists, it's called Google auto-suggest." But I specifically want it to work when I'm not (yet) actively searching, but just writing. (If it already existed, it might have stopped me from writing this blog post. :-) Twitter might also become a different place if users realized how many others have already entered the same item.</div><div><br /></div><div>Of course there's a little privacy issue. But still, if this existed, I'd opt in! (In fact, I did half a dozen searches while I was typing this. How much easier it would be if I didn't have to select text, switch to a different tab, paste, and hit enter, losing my writing context each tim.)</div></div>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com25tag:blogger.com,1999:blog-4195135246107166251.post-66864086065089448882009-11-05T10:12:00.000-08:002009-11-07T11:48:12.618-08:00Python in the Scientific World<div><div>Yesterday I attended a biweekly meeting of an informal a UC Berkeley group devoted to Python in science (Py4Science), organized by Fernando Perez. The format (in honor of my visit) was a series of 4-minute lightning talks about various projects using Python in the scientific world (at Berkeley and elsewhere) followed by an hourlong Q&A session. This meant I didn't have to do a presentation and still got to interact with the audience for an hour -- my ideal format.</div><div><br /></div><div>I was blown away by the wide variety of Python use for scientific work. It looks like Python (with extensions like numpy) is becoming a standard tool for many sciences that need to process large amounts of data, from neuroimaging to astronomy.</div><div><br /></div><div>Here is a list of the topics presented (though not in the order presented). All these describing Python software; I've added names and affiliations insofar I managed to get them. (Thanks to Jarrod Millman for providing me with a complete list.) Most projects are easily found by Googling for them, so I have not included hyperlinks except in some cases where the slides emphasized them. (See also the blog comments.)</div><div><ul><li>Fernando gave an overview of the core Python software used throughout scientific computing: NumPy, Matplotlib, IPython (by Fernando), Mayavi, Sympy (about which more later), Cython, and lots more.</li></ul></div><div><ul><li>On behalf of Andrew Straw (Caltech), Fernando showed a video of an experimental setup where a firefly is tracked in real time by 8 camaras spewing 100 images per second, using Python software.</li></ul></div><div><ul><li>Nitimes, a time-series analysis tool for neuroimaging, by Ariel Rokern (UCB).</li></ul><ul><li>A comparative genomics tool by Brent Pedersen of the Freeling Lab / Plant Biology (UCB).</li></ul><ul><li>Copperhead: Data-Parallel Python, by Bryan Catanzaro (working with Armando Fox) and others.</li></ul><ul><li>Nipype: Neuroimaging analysis pipeline and interfaces in Python, by Chris Burns (http://nipy.sourceforge.net/nipype/).</li></ul><ul><li>SymPy -- a library for symbolic mathematics in Pure Python, by Ondrej Certik (runs on Google App Engine: http://live.sympy.org).</li></ul><ul><li>Enthought Python Distribution -- a Python distro with scientific batteries inluded (some proprietary, many open source), supporting Windows, Mac, Linux and Solaris. (Travis Oliphant and Eric Jones, of Enthought.)</li></ul><ul><li>PySKI, by Erin Carson (working with Armando Fox) and others -- a tool for auto-tuning computational kernels on sparse matrices.</li></ul><ul><li>Rapid classification of astronomical time-series data, by Josh Bloom, UCB Astronomy Dept. One of the many tools using Python is GroupThink, which lets random people on the web help classify galaxies (more fun than watching porn :-).</li></ul><ul><li>The Hubble Space Telescope team in Baltimore has used Python for 10 years. They showed a tool for removing noise generated by cosmic rays from photos of galaxies. The future James Webb Space Telescope will also be using Python.<span><span> (Perry Greenfield and Michael Droettboom, of STSCI.)</span></span><span class="Apple-style-span" style="font-family: arial, sans-serif, 'Arial Unicode MS'; font-size: 13px; border-collapse: collapse; "></span></li></ul><ul><li>A $1B commitment by the Indian government to improve education in India includes a project by Prabhu Ramachandran of the Department of Aerospace Engineering at IIT Bombay for Python in Science and Engineering Education in India (see http://fossee.in/).</li></ul><ul><li>Wim Lavrijsen (LBL) presented work on Python usage in High Energy Physics.</li></ul><ul><li>William Stein<span><span> (University of Washington)</span></span><span class="Apple-style-span" style="font-family: arial, sans-serif, 'Arial Unicode MS'; font-size: 13px; border-collapse: collapse; "></span> presented SAGE, a viable free open source alternative to Magma, Maple, Mathematica and Matlab.</li></ul></div><div>All in all, the impression I got was of an incredible wealth of software, written and maintained by dedicated volunteers all over the scientific community.</div><div><br /></div><div>During the Q&A session, we touched upon the usual topics, like Python 3 transition, the GIL (there was considerable interest in Antoine Pitrou's newgil work, which unfortunately I could not summarize adequately because I haven't studied it enough yet), Unladen Swallow, and the situation with distutils, setuptools and the future 'distribute' package (for which I unfortunately had to defer to the distutil-sig).</div><div><br /></div><div>The folks maintaining NumPy have thought about Python 3 a lot, but haven't started planning the work. Like many other projects faced with the Python 3 porting task, they don't have enough people who actually know the code base well enough do embark upon such a project. They do have a plan for arriving at PEP 3118 compliance within the next 6 months.</div><div><br /></div><div>Since NumPy is at the root of the dependency graph for much of the software packages presented here, getting NumPy ported to Python 3 is pretty important. We briefly discussed a possible way to obtain NumPy support for Python 3 sooner and with less effort: a smaller "core" of NumPy could be ported first, which would give the NumPy maintainers a manageable task, combined with the goal of selecting a smaller "core" which would give them the opportunity for a clean-up at the same time. (I presume this would mostly be a selection of subpackage to be ported, not an API-by-API cleanup of APIs; the latter would be a bad thing to do simultaneous with a big port.)</div><div><br /></div><div>After the meeting, Fernando showed me a little about how NumPy is maintained. They have elaborate docstrings that are marked up with a (very light) variant of Sphynx, and they let the user community edit the docstrings through a structured wiki-like setup. Such changes are then presented to the developers for review, and can be incorporated into the code base with minimal effort.</div><div><br /></div><div>An important aspect of this approach is that the users who edit the docstrings are often scientists who understand the computation being carried out in its scientific context, and who share their knowledge about the code and its background and limitations with other scientists who might be using the same code. This process, together with the facilities in IPython for quickly calling up the docstring for any object, really improves the value of the docstrings for the community. Maybe we could use something like this for the Python standard library; it might be a way that would allow non-programmers to help contribute to the Python project (one of the ideas also mentioned in the diversity discussions).</div></div>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com35tag:blogger.com,1999:blog-4195135246107166251.post-72783970711392577862009-09-17T09:48:00.000-07:002009-09-17T10:00:58.497-07:00Lovely Python!I just heard from Bill Xu in China. His book "Lovely Python", an introduction to Python in Chinese, was just published and shot to the top-5 of <a href="http://www.china-pub.com/">china-pub.com</a>'s bestseller list (at some point it even was #2). I can't read Chinese, but I am very glad that there's a book on Python available for Chinese readers, and that's why I wrote a brief foreword for the book as well. (Also because I am one of the mentors of <a href="http://zeuux.org/">Zeuux.org</a>.) Links:<div><br /></div><div>This is the Lovely Python page on the book store:</div><div><a href="http://www.china-pub.com/195771">http://www.china-pub.com/195771</a></div><div><br /></div><div>This is the bestseller list:</div><div><a href="http://www.china-pub.com/rank/?type=59&act=day&v=7">http://www.china-pub.com/rank/?type=59&act=day&v=7</a></div><div><br /></div><div>This is the cover (with front and back) of Lovely Python:</div><div><a href="http://www.zeuux.org/pub/lovely-python-cover.jpg">http://www.zeuux.org/pub/lovely-python-cover.jpg</a></div><div><br /></div><div>This is my preface:</div><div><a href="http://www.zeuux.org/science/zeuux-lovely-python-preface-by-guido.html">http://www.zeuux.org/science/zeuux-lovely-python-preface-by-guido.html</a> (English)</div><div><a href="http://www.zeuux.org/science/zeuux-lovely-python-preface-by-guido.cn.html">http://www.zeuux.org/science/zeuux-lovely-python-preface-by-guido.cn.html</a> (Chinese translation by Bill Xu)</div><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_FG9t5W1SJ14/SrJq4ymLxBI/AAAAAAAAEOA/P9IP7JQPKcw/s1600-h/zcover.jpg"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 176px; height: 200px;" src="http://3.bp.blogspot.com/_FG9t5W1SJ14/SrJq4ymLxBI/AAAAAAAAEOA/P9IP7JQPKcw/s320/zcover.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5382482028548310034" /></a>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com9tag:blogger.com,1999:blog-4195135246107166251.post-28742476590607822992009-07-22T13:29:00.000-07:002009-07-22T18:03:19.999-07:00Scientists Discover That Hidden Persuaders Are RealIn <a href="http://neopythonic.blogspot.com/2009/07/progressive-vs-conservative.html">yesterday's post</a> I mentioned reading <a href="http://en.wikipedia.org/wiki/George_Lakoff">George Lakoff</a>'s book, The Political Mind. While I agree with the politics of the book in almost every instance, I was still disappointed. For one thing, the book "compresses well." (IOW it contains a lot of repetition. A Lot.) It also felt a bit like a classic bait-and-switch: the back flap touts "the science behind how our brains understand politics" but the contents are 90% political rhetoric, and I'm still in doubt about the science.<br /><br />The author's premise is attractive enough as far as it goes: our brains don't make perfectly rational decisions, but are influenced by "framing"; and the Republicans have used this to their advantage while the Democrats with their belief in "pure reason" have not properly defended themselves by accepting the conservative framing (for example: "tax relief").<br /><br />Well, there may be some recent scientific research that confirms that most people are not so good at rational decision making, but honestly, I thought that the importance of framing has been well known for a long time to all politicians -- and <a href="http://en.wikipedia.org/wiki/Vance_Packard#The_Hidden_Persuaders">advertisers as well</a>. As far as the recent scientific proof for this commonly-known fact, <a href="http://www.jonahlehrer.com/books">Jonah Lehrer</a>'s book "How We Decide" <a href="http://www.jonahlehrer.com/books"></a>contains at least as much about the research, and the non-scientific parts of his book are better written and, I expect, more future-proof.<br /><br />I'm also skeptical of the importance of Lakoff's discovery that frames are represented <span style="font-style: italic;">physically</span> in the brain. That's about as insightful as saying that this blog entry exists physically in Google's computers (as magnetic fluctuations on a hard drive). Has he never heard of abstractions? He seems to argue that all of philosophy needs to be thrown away because it ignores this fact. I will gladly accept that we cannot treat the brain as a perfect mathematical machine, and using the embodyment of the mind will probably eventually help us understand consciousness (more likely than abstract reasoning like <a href="http://www.cogs.indiana.edu/people/homepages/hofstadter.html">Douglas Hofstadter</a>'s approach, no matter how much I enjoy his puzzles and paradoxes).<br /><br />But the important message to me is still about how the brain's <span style="font-style: italic;">software</span> works. It's useful to know that frames are reinforced by trauma and repetition, and that it requires a lot of repetition of counteracting frames to override them once they're there. And yes, that the Bush government used this to its advantage is a great example. But I wanted to know more about the science, and less about the politics.<br /><br />Lakoff's other point is that human beings are born to have emphathy with each other. But he doesn't mention much of the science behind this. That's because in the end he is a linguist, and linguists spend most of their time studying (and arguing about) human language, which was the <span style="font-style: italic;">result</span> of a long evolutionary path and cannot necessarily <span style="font-style: italic;">explain</span> it. And his oft-repeated use of the words America and American in connection to empathy is surely his own little joke, where he's trying to make the reader believe that American values are nurturing values by applying his own theory: say it over and over and the frame will be hard-wired (whether that's literally or figuratively :-) in the reader's mind. As a non-US-citizen I wished the emphasis was on human values, not American values.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com10tag:blogger.com,1999:blog-4195135246107166251.post-60514488945355984812009-07-21T11:50:00.000-07:002009-07-22T13:29:10.128-07:00Progressive vs. Conservative[Warning: loose thoughts ahead!]<br /><br />Microsoft's Eric Meijer gave a talk at Google yesterday, and afterwards I had lunch with him. One of his remarks was (I paraphrase) that Microsoft users want to be told what to do, while the Java community is more vocal or argumentative. (He didn't discuss the Python community but in my experience it falls in the latter category.)<br /><br />Now, while lying sick in bed with a hacking cough, I am reading George Lakoff's "The Political Mind". This book tries to model the distinction between conservative and progressive politics on the differences between two different ideal family models: the strict father (from which most conservative moral virtues flow according to Lakoff), and the nurturing family, from which the progressive moral virtues derived.<br /><br />The parallel with Microsoft users vs. Java users seems to be all too obvious: Microsoft as the strict father: If you are loyal you will be rewarded, but if you stray you will be punished; whereas in the Java (or Python) community benefits and moral goodness flow from helping each other (which includes sharing open source software, and, apparently, bikeshedding :-).<br /><br />What about other companies and communities? I can't help thinking of Oracle as the ultimate strict-father company, which makes me worry about the Sun takeover. Are Linus Torvalds and Richard Stallman strict fathers?Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com26tag:blogger.com,1999:blog-4195135246107166251.post-81171063899703871492009-06-26T12:05:00.000-07:002009-06-26T17:20:39.049-07:00IronPython in Action and the Decline of WindowsWhile CPython still rules on python-dev, especially with the excitement around Py3k, Python's alternative implementations are growing up: PyPy is now capable of running Django, Jython just released version 2.5, and IronPython has been releasing significant milestones like clockwork. I get a lot of satisfaction out of such milestones: they help establish Python as a language you can't ignore, no matter which platform you are using.<br /><br />Seeing a book like <a href="http://www.manning.com/foord/">IronPython in Action</a>, by Michael Foord and Christian Muirhead, is another milestone for IronPython. This is a solid work in every aspect, and something nobody using IronPython on .NET should be without. The book is chock full of useful information, presented along with a series of running examples, and covers almost every aspect of IronPython use imaginable.<br /><br />After reading the table of contents and the introduction by IronPython's creator Jim Hugunin, I couldn't help myself and skipped straight to appendix A, "A whirlwind tour of C#." This is a useful thing to have around for readers like myself who haven't really kept track of things in the .NET world. Maybe I'll comment more on C# another time. For now, let me just say that it seems a decent enough system programming language. The more relevant thing about C# is that you can't avoid learning it if you are developing on .NET, even when using IronPython. There just are too many issues where IronPython has to work around a limitation of C#. This happens often indirectly where a particular API was designed purely with C# in mind. And then there's the issue that Microsoft's API documentation focus on C#. (And VB.NET, I suppose, which after seeing some samples of in this book I have less desire to know than ever.)<br /><br />There are some introductory chapters -- some fluff about .NET and the CLR, an introduction to Python, and an introduction to with .NET objects from IronPython. The Python introduction has a slight emphasis on differences between IronPython and CPython, though there aren't enough to fill a chapter. This is a good thing! The chapter does a pretty good job of teaching Python, assuming you already know programming. In general, the book is aimed solidly at professional software developers: unless you are paid to do it, why would anyone want to get intimate with Windows?<br /><br />Yes, Windows programming is what this book is really about. I'm sure that doing Windows programming using IronPython is a much better proposition than Windows programming using C++; but it's still Windows programming. Fortunately the authors maintain a slightly ironic attitude about Windows. I can't help admiring their persistency in getting to the bottom of the many mysteries presented by Windows(and in some cases by IronPython's wrappers).<br /><br />Many, many years ago -- so long ago that I can't even recall when -- I did some Windows programming myself, using Mark Hammond's Win32 extensions for CPython. That package maps the Win32 API pretty directly to Python. It lets you work with Windows in much the same way as you can in IronPython -- the main difference is that IronPython lives in the more modern .NET world, while Win32 is showing its age.<br /><br />But is life with IronPython all that much better than in CPython+Win32? It still looks incredibly tedious to create the simplest of UI. Each button in the UI has to be tediously positioned and configured (width, height, padding, font size, etc.). The book maintains a running example throughout many of the chapters, and one of the earlier versions (with many features not yet developed) clicks in at a "mere" 258 lines. Fortunately, the source code for all examples can be downloaded from the book's website. While the downloaded zip file is a whopping 33 megabytes, there's actually only half a megabyte of source code in it (and much of it multiple versions of the same running example) -- the majority of the download is not source code but DLLs that are probably included by the authors because Microsoft scatters them around half a dozen or more different support websites. (Plus, there seem to be multiple copies of IronPython.dll and a few other DLLs included.)<br /><br />This then, was the big eye-opener for me: that despite all the hype, Windows UI programming is as tedious today as it was in 1995. Sure, the new UI looks a lot better. But that's mostly glitz: 3D effects, color gradients, video, and so on. Sure, it's all object-oriented now. But it hasn't really gotten any less complex to create the simplest of simple UIs. And that's a shame. When is Microsoft going to learn the real lesson about simplicity of HTML? Instead, Microsoft is doing the same thing to HTML that it does to anything it touches: adding cruft to the point where the basic functionality is buried so deeply that most people can't even find it. You can't really blame the average Windows developer for focusing on eye candy instead of usability -- it's all mashed together in the APIs, and by the time you've got something that works at all, you're too exhausted to look at it from your users' perspective.<br /><br />It's no wonder that users are switching to the web as the platform for everything that used to live on the desktop -- with all its flaws (which I will discuss another time), web development still feels like a breeze compared to Windows development. And that means less time to release, and hence more frequent releases, which in turn means more opportunities for developers to learn what their users actually do. Which as a user I really appreciate.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com48tag:blogger.com,1999:blog-4195135246107166251.post-5663924198735688502009-06-16T18:47:00.000-07:002009-06-16T20:45:18.074-07:00Highs and Lows of IEEE Computer MagazineI still read a few print publications, including IEEE Computer. Today's issue contained a high and a low:<br /><br />Today's high point was a detailed history of the Conficker worm. Since we're a Macintosh family, and Google typically has its security stuff in order, I was barely aware of it. The sophistication of the worm's creators is almost admirable. (They probably use Python too. :-) An interesting table in the article included information about which countries contribute the most to the worm's population. China, Brazil and Russia top the list. You could have all sorts of theories on why this would be; personally I'm assuming it's a combination of sheer number of computers plus widespread use of bootlegged copies of Windows.<br /><br />The low point was an article on "Software Engineering Ethics." Why a low point? Look at this table and think of how many bits of information it contains:<table><br /><tbody><tr><th colspan="3">Using postphenomenology for software engineering ethics</th></tr><br /><tr><th>Actions</th><th>Desirable</th><th>Undesirable</th></tr><br /><tr><td>Amplify experiences that are</td><td>+</td><td>-</td></tr><br /><tr><td>Reduce experiences that are</td><td>-</td><td>+</td></tr><br /><tr><td>Invite actions that are</td><td>+</td><td>-</td></tr><br /><tr><td>Inhibit actions that are</td><td>-</td><td>+</td></tr><br /></tbody></table><br />Ironically, this pointless table contains a redundant column, while the table I mentioned above was missing a column that would have been useful -- how many PCs are installed in each country. Oh well.<br /><br />PS: Googling for "postphenomenology" gives this as the title of the first hit: "If phenomenology is an albatross, is postphenomenology possible?" The web knows best.Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com4tag:blogger.com,1999:blog-4195135246107166251.post-77265098726101938292009-06-15T09:20:00.000-07:002009-06-15T10:07:33.386-07:00New App Engine BookAt Google I/O I received a copy of <i>Using </i><i>Google App Engine</i> by Charles Severance, published by O'Reilly. I haven't kept track, but this appears one of the first App Engine books to actually hit the stores -- an Amazon search for App Engine showed up one other book (<i>Developing with Google App Engine</i> by Eugene Ciara, published by APress) and many titles available for pre-order (including additional titles from the same publishers).<div><br /></div><div>Severance's book is a quick read if you're already familiar with the basic premises of web programming. I think it would do well in an introductory course about the topic. (The author teaches at the University of Michigan so this is likely how he developed the material in the first place.) In fact, quite a bit of the book could well have come from a pre-existing earlier course: the chapters on HTML, CSS, Python and JavaScript barely mention App Engine.</div><div><br /></div><div>Don't get me wrong, I think that's a good approach: in my experience quite a few App Engine users are new to web programming in general, or could at least use a refresher course. If you don't fall in this category, don't feel offended: you just probably aren't the intended audience for this book. On the other hand, if you've developed for the web but haven't used Python before, you could probably just skip the HTML/CSS chapter and dive right into Python and App Engine.</div><div><br /></div><div>If you're a blank sheet where it comes to programming, don't expect to come out an experienced Python developer: the book only covers enough of the language so you can get started with App Engine without feeling you're just copying and pasting text. The same is actually true for any topic covered -- in many cases the book actually recommends that you study a topic more in-depth using other resources. But in each case the book's coverage is enough to get you started with the creation of dynamic web sites, and that's the important part. After all, you didn't learn your mother tongue by studying the rules of grammar either: you learned a few nouns, a few verbs, a few adjectives, and a few grammatical forms ("Daddy throw toy again") and you were on your way to communicating with others.</div><div><br /></div><div>Actually, if you read this book from cover to cover, you might not be ready to create the Greate American Website, but you'll be well past the "Daddy throw toy" level. For example, you'll be creating App Engine datastore models with ease, tying them together with forms, and you'll even be able to use simple AJAX patterns. You will also have learned about the importance of caching, and you'll have more than a fleeting experience debugging problems using tracebacks and logs.</div><div><br /></div><div>I also enjoyed some of the history bits that Severance presents (it makes me feel old to see 1990 referred to as ancient history :-). A downside is that sometimes the exercises given at the end of each chapter seem to be focused more on assessing that you were awake during class than that you actually have learned a useful skill (e.g. "Give a brief history of the major phases of the internet"). Teachers considering to use this book in the classroom might appreciate such questions; but for self-study, I would focus on the difference between the class= and id= attributes in HTML...</div><div><br /></div><div>What's missing? The book doesn't touch Django (except for the templating facility built into App Engine's webapp package, which is based on Django). If our customer support traffic is any indication, Django is very popular with professional App Engine developers. The book also doesn't describe the various APIs offered by App Engine for things like sending mail, fetching other web resources by URL, or image processing. But arguably you can learn those directly from the App Engine docs. Oh, and the book doesn't touch on App Engine's Java support. I expect other books will fill that void.</div>Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com12tag:blogger.com,1999:blog-4195135246107166251.post-32857474489422012502009-05-26T15:46:00.000-07:002018-09-06T08:03:14.187-07:00So you want to learn Python?There's never a lack of books to use for learning Python. I occasionally receive books for review, but I don't have a particularly good yardstick to judge such books by: I find that they all contain some factual errors and some oddities of presentation, but I have no idea whether those matter for the readers. Even Knuth's books are full of errors: for example <a href="http://www-cs-faculty.stanford.edu/%7Euno/taocp.html">the errata for Vol. 1 (2nd ed.) are a staggering 80 pages</a>, but I doubt anybody besides Knuth himself is bothered by this knowledge.<br /><br />Recently I got a review copy of "Hello World", and a colleague kindly lent me his copy of "Practical Programming". I think it's interesting to compare the two a bit, since they both claim to be teaching Python programming to people who haven't programmed before. And yet their audiences are totally different!<br /><br />"Hello World", published by Manning, is written by Warren Sande and his son Carter. The subtitle is "Computer programming for kids and other beginners", but I think if you're not a kid any more you might get annoyed by the rather popular writing style. If you <span style="font-style: italic;">are</span> a kid, well, you will probably enjoy a book written with you in mind, and you will learn plenty. The only prerequisites are reading and typing skills, a computer that wasn't built in the stone age, and a desire to learn more about what goes on inside that computer. The book uses short chapters with lots of illustrations, often cartoons and jokes. There are lots of opportunities to try out the material and learn that way. Each chapter ends with a review section, some tests, and more experiments to try. The book pays plenty of attention to typical "gotchas", so that if you get stuck at some point, there probably is help nearby to get you unstuck.<br /><br />"Practical Programming" is written by Jennifer Campbell, Paul Gries, Jason Montojo, and Greg Wilson. This a team composed of three university professors and a former student of theirs. Their purported goal is to teach Computer Science (with Capital Letters), and Python is merely a teaching vehicle. But they spend about half of the book on Python itself, covering roughly the same material as any introduction to Python, including "Hello World". Their intended audience is clearly more mature than that of the Sandes, and I would think that Carter Sande and his friends would have a hard time staying focused on the material as presented by Campbell et al. -- their illustrations and diagrams are more functional but a lot less fun.<br /><br />Both books present a number of projects and running examples. Again, the difference in audience makes it likely that if you love one, you'll hate the other, and vice versa. "Hello World" uses examples from computer games. The games are extremely simple though: modern computer games are some of the most complex system around, and you can't expect to approach them using PyGame and a couple hundred lines of Python. "Practical Programming" takes its example from scientific data processing with an environmental touch: for example, a numerical series is presented as whale sightings over the years and 2-dimensional data is taken from deforestation data. No doubt this is done in an attempt to appeal to a certain kind of student, though the number of potential applications is so large that some students might just as well be turned off by the specific set of choices.<br /><br />In the end, "Hello World" will leave the reader with a fair amount of practical Python experience, enough to get them started on the long road to becoming a programmer if they are so inclined, or at least enough to give them some idea of what it is that programmers do. "Practical Programming" tries to go further: it presents some well-known algorithms (there's even a discussion of MergeSort), and it has introductory chapters on topics like object-oriented programming and databases. The overall focus is still on being able to use all this new knowledge in one's professional life, and I hesitate to agree with the authors' apparent view that it teaches "Computer Science". Calling it "Computer Use" would cover the contents better, I think, and that's more in line with the series title as well ("The Pragmatic Programmers", also the publisher).<br /><br />So, how <span style="font-style: italic;">do</span> you learn about Computer Science? Some would no doubt recommend "Structure and Interpretation of Computer Programs" by Abelson and Sussman here. (Someone sent me review copy of that book too.) But really, SICP (as it is often referred to) has its own agenda: convincing the reader that the most important thing computers can do is interpreting computer programs. This agenda has arguably caused the proliferation of Scheme implementations and indoctrinated many young minds with certain ideas about how to design and implement programming languages. But personally, I recommend you go straight to <a href="http://www-cs-faculty.stanford.edu/%7Euno/taocp.html">the source</a>. After all these years, there is still no substitute for Knuth.<br /><br />[UPDATE: fixed book titles as commenters pointed out my typos.]Guido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.com32