Paxos Engineering Blog

Python 3's Killer Feature: asyncio

A massive debate in the python community about python2/3 has been raging for years. The main reason for python3 to diverge was to provide unambiguous types to handle unicode, strings and bytes (more here), but recently there’s been a bigger divergence that’s gone largely unnoticed. python3.4 introduced the asyncio module and python3.5 gave it a new syntax that is built into the language. Asyncio has done for python what node did for javascript. Asynchronous programming in python enables you to do new and powerful things, but like all technologies there are tradeoffs.



We use uvloop to speed up our asyncio code (link):

“uvloop makes asyncio fast. In fact, it is at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework. The performance of uvloop-based asyncio is close to that of Go programs.”

Asyncio can significantly improve throughput on the same hardware. You don’t have to use semaphores, you get access to shared memory, and it’s relatively easy to code. However this won’t improve latency since await-ed functions still have to wait for IO. While our await-ed functions wait for IO, control is automatically returned to the event loop so other code can run.

For applications with lots of IO, the savings can be substantial. Imagine a service like pingdom that visits a ton of websites to be sure they’re still up. Serially making HTTP requests and waiting for responses would be painfully slow. Before asyncio, you had to use python’s threading module, which has several limitations.


Example Code

Let’s start, by showing what some asynchronous python 3.5+ code looks like:


Regular Snippet 

async def get_popular_words(words_to_ignore, histogram_size):

   # make asynchronous call
   data = await slow_function_that_waits_for_io()

   # calculate histogram for data
   histogram = {}
   for word in data.split():
       if word not in words_to_ignore:
           histogram[word] = histogram.get(word, 0) + 1

   # return only top `histogram_size` most common words (and frequency)
   to_return = {}
   for cnt, word in enumerate(sorted(histogram, key=histogram.get, reverse=True)):
       to_return[word] = histogram[word]
       if cnt = histogram_size-1:

   return to_return


At first glance, it looks strange (you might say unpythonic!) and is a bit harder to wrap your head around. But actually, it’s just a bunch of normal looking python code with two differences. One is an async in front of the get_histogram method definition, and another is an await in front of the asynchronous slow_function_that_waits_for_io function call.

What’s messier is how to run this code:

Running Your Async Code 

import asyncio
from my_module import import get_popular_words
if __name__ == '__main__':
   coroutine = get_popular_words(
       words_to_ignore=set(['the', 'and', 'to']),
   popular = asyncio.get_event_loop().run_until_complete(coroutine)


If you’re using a framework like aiohttp or sanic you’ll just follow the default instructions and not have to think about this much. If you dig into the details you’ll have to learn all about coroutines and what’s happening under the hood.



While this performance sounds great, what does it cost? We’ve found programming with asyncio to have some downsides.


Hard to Use the Interpreter

One of the nice things about python is that you can always fire up the interpreter and step through code easily. For asyncio code, you need to run your code using an event loop. In practice, this usually means adding a bunch of print statements to whatever isn’t working and re-running your code.


All or Nothing

It’s hard to just use asyncio for the few method(s) that would most benefit from the performance, since you need to run your code with an event loop. You’ll have to jump into the deep end!

In python3’s early days, there was valid concern about library support. Today, python3 has overtaken python2 in library support. However, asyncio is trickier. For example, the popular requests library has chosen to not be asyncio compatible (note: aiohttp is a great asyncio compatible alternative). You’ll probably need a new database adaptor.


You Have to Think about the Event Loop

Are you awaiting the result of several actions and then performing some action on that data? If so, you should be using asyncio.gather. Or, perhaps you want to asyncio.wait on future completion? Do you want your future to run_until_complete or run_forever?

Did you forget to put async in front of a function definition that uses await? That will throw an error. Did you forget to put await in front of an asynchronous function? That will return a coroutine that was never invoked, and intentionally not throw an error!

asyncio doesn’t play that nicely with threads, so if you’re using a library that depends on threading (like Cassandra’s python driver) you’re going to have to come up with a workaround.


Everything is Harder

Libraries are often buggier and less used/documented (though hopefully that will change over time). You’re not going to find as many helpful error messages on Stack Overflow. Something as easy as writing asynchronous unit tests is non-trivial.

There’s also more opportunities to mess up. For example, when we moved one app to asyncio we forgot to move over a few synchronous API calls that were using the requests library. Since we were running this app on a single thread, those ~100ms API calls were making the whole app hang! It took a while to figure out why.


When should I use asyncio?

It depends! With more python3 library support than python2, there’s no good reason to build a new app using python2. It makes sense to use asyncio if your app wastes a lot of cycles waiting on IO, is a good fit for an asynchronous framework (especially websockets), and resource intensive (reduce your server bill).



This blog post was inspired by Jimmy Song’s recent talk at the NY Python Meetup. Check it out for a more in-depth discussion of asyncio.

Interested in working with these technologies (and more)? We’re hiring software engineers.


Podcast: How Private Blockchains Can Streamline Capital Markets

This post originally appeared on the Repository.

There has been a lot of discussion around the value of financial applications of blockchain technology and where they are applicable in the industry. At Paxos, we are laser-focused on modernizing and enhancing post-trade settlement which we feel can significantly streamline how capital markets operate.

This 30-minute FutureTech podcast explains how private blockchains can improve post-trade settlement processes and discusses our Euroclear Bankchain blockchain gold settlement service for the London bullion market, set to launch later this year.

Why Take-Home Tests Are Awesome

Written by Raj Nair, VP of Engineering, with contributions from Michael Flaxman, Principal Engineer


At Paxos, we give all software engineering candidates a short take-home test before deciding if we want to move forward with an in-person interview. We’ve found it works really well, so we want to share our process and why we think it’s so effective.

Our hiring process in one easy flow: phone interview, take-home test, in-person interview, and offer!


We don’t just give out take-home tests for our own benefit; it saves candidates time as well. Our offer rate after in-person interviews is much higher than if we gave interviews to everyone who passed a basic phone screen; we’re often able to confidently give candidates a yes-or-no answer after only a few hours of in-person interviews. The take-home test eliminates the need to do marathon 6-hour interview days or multiple rounds of interviews.

The biggest advantage of the take-home test, however, is that candidates have the opportunity to showcase their best work under more realistic working scenarios. Our take-home test can be completed on the candidate’s schedule, which is especially useful if their full-time job keeps them busy during work hours. More importantly, we know that writing code live can be nerve-wracking, and it can be hard to do a great job under pressure. If someone aces an in-person code test they’re probably very talented, but conversely if they do a mediocre job, does that mean their skills are mediocre? Perhaps the question was explained in a confusing way, the candidate was thrown off by working on a foreign development environment, exhausted from a day of interviews, tired from a bad night of sleep, freezes on writing code on a whiteboard. The list goes on.

In a competitive market for software engineers, we can’t afford to only hire candidates with perfect in-person interviews. Human communication is a lossy protocol!

When we grade a submission, we’re evaluating it like we would a pull request on our codebase; we expect it to be great! We always make it clear to candidates that we want them to show off their best work. Two engineers grade each submission to ensure we are both thorough and fair. Here are some common things that tend to cause poor code reviews:


PerformanceIs your code performant?

Every question we ask has ideal, acceptable and bad performance. One benefit of the take-home is that you have plenty of time to think about performance and revise your work. There’s no excuse for an O(n2) answer to an O(n) problem. Bonus points go to a candidate who explains the performance of their code and how they could improve it if needed.


engineering iconREC-09.pngDid you give yourself a code review before you submitted your code?

Since code is generally read many more times than it is written, we expect all code at Paxos to be self-reviewed before submission. Often times, candidates try to solve a problem one way and then change their approach along the way. That’s totally normal, but it’s important to go back and clean up the code so that it makes the most sense given how you ultimately solved the problem. Leaving in unused variables, commented out code and print statements for debugging demonstrates a lack of caring for your craft.


engineering iconREC-10.pngDid you use the right tool for each problem?

There’s often multiple correct ways to do something, but you should pick your tools carefully and then use them in the way that best demonstrates their utility.


engineering iconREC-11.pngDoes your code flow logically?

This one is hard to define, but you know it when you see it. Generally, shorter code is better code, but this heuristic has a limit. We sometimes get submissions that contain 100 files to solve a problem that can be solved in 30 lines of code (the former may be acceptable if well documented).


engineering iconREC-12.pngAre variables and methods well named?

It’s one of the three hard problems in computer science.


engineering iconREC-13.pngDid you include clear and concise documentation?

We accept submissions in any language, so wherever there is ambiguity you should be clear about versions, dependencies, platforms and other configuration instructions.


engineering iconREC-14.pngDid you include a working example and/or test coverage?

If your submission is perfect, this may not be strictly necessary. But if you made any mistakes, it’s easy to forgive a minor error if you can easily see what the candidate was trying to do.


engineering iconREC-16.png

Is your use of commenting appropriate?

Unclear parts of your code should be well commented (or better yet, made clear enough so that they don’t need comments), and obvious parts of your code should not be over-commented.


engineering iconREC-15.png


Each question we ask includes example inputs/outputs. It’s amazing how many submissions we get that don’t meet the spec.

Sometimes candidates do an excellent job on their take-home, but when they come in for an in-person interview they're unable to solve a simple fizzbuzz problem. To combat cases of cheating (most commonly, getting help from friends), we do two things:

  1. Before flying candidates in from other cities, we ask them to solve a simple live-coding question on a Google Hangout. This won’t tell us that they’re extremely talented, but it guarantees a minimal ability level.

  2. During their interview, we ask candidates about their code. If they’re unclear how it works, it’s a major red flag. We sometimes ask them to improve their code on the spot or to build on it to solve other problems. We think it’s fair game to ask more challenging in-depth questions since the candidate has previously had time to think through the problem.

We care about many things when evaluating a candidate, but the most important thing we’re testing in the interview process is work product. There is no better approximation than doing a test that simulates real work, especially when it’s on your own time and in the comfort of your own space. Our problems are designed to simulate the types of challenges you’d be working on at Paxos so we can see how you would think through them.

The strongest objection we get from candidates is that they’re too busy at work and don’t have time for this. Our take-home is short, so that shouldn’t hold you back. We currently ask three questions, each are solved on average in 50 lines of code (some can be done in much fewer!). They’re designed to be fun problems, so if you enjoy writing code you should enjoy the challenge. We’ve been iterating for 9 months on our choice of questions, wording and examples, so we think it’s pretty good.

Though take-home tests aren’t a silver bullet, they’ve been really helpful for us at getting a fuller picture of a candidate’s abilities while saving everyone time in the process. Candidates seem to enjoy the questions, too!


Want to try your hand at our take-home test? We're hiring engineers across multiple roles, and each role has its own take-home test.



Write Fast Apps Using Async Python 3.6 and Redis

One of the common complaints people have about python and other popular interpreted languages (Ruby, JavaScript, PHP, Perl, etc) is that they’re slow. A faster language would give better response times and reduce server costs.

python3.4 introduced the asyncio module and python3.5 gave it a new syntax that is built into the language. Then, along came uvloop with a huge speed increase:

uvloop makes asyncio fast. In fact, it is at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework. The performance of uvloop-based asyncio is close to that of Go programs.

At Paxos, we make heavy use of asyncio because it’s more performant and also because it’s a better fit for our architecture. Our product synchronizes transaction data from many different clients and is inherently asynchronous because it’s a distributed system.

asyncio is great for maximizing throughput, but we also care about reducing latency. For latency, your database can be a major bottleneck. When using best practices like a proper index and SSDs, a ~10 ms response time for a database query is considered good.

RAM is orders of magnitude faster, but simple key-value stores don’t allow for many of the features software engineers rely on. Multiple indices and basic type checking are very useful. ORDER BY, LIMIT, and OFFSET, are nice to have as well.

We’ve been impressed with redis due to its performance, advanced features and documentation. So, we wrote a redis-backed object mapper called subconscious that relies on asyncio and redis’ powerful primitives. You get the benefits of a database, with the performance of RAM! It’s like a bare-bones version of SQLAlchemy that will never support JOIN operations.

You can define a model like this (for more details, check out the official repo):

class Gender(enum.Enum):
   MALE = 'male'
   FEMALE = 'female'

class User(RedisModel):
   uuid = Column(primary_key=True)
   name = Column(required=True)
   age = Column(index=True, type=int, sort=True, required=True)
   gender = Column(index=True, enum=Gender)
   country_code = Column(index=True)


Then you can query and use your model like this:

some_user = await User.load(db, some_uuid)
if some_user.age > 65:
   # apply senior citizen discount
   price -= 10.00


You’ll also find lots of features you’ve come to expect from traditional ORMs. Here’s how you’d send a promotion to your US, college-age, male users:

async for user in User.filter_by(
   age=[18, 19, 20, 21, 22],
   await send_promo_email(user)


In order to demonstrate how powerful and easy to use this is in practice, we’ve thrown together a pastebin-like website in ~100 lines of code: (source code here)


We load-tested this app on Microsoft Azure’s cheapest instance (called an F1, which has 1 core and 2 GB RAM), and the numbers are great:

  • 10k pageviews took ~41s
  • Median pageview was ~19ms
  • 90% of pageviews return in < 22ms

Redis can be saved to disk, but has much weaker persistence vs a traditional database. Transactions and rollbacks are complicated, and there are less guarantees when it comes to data integrity. Some common use-cases:

  • Ephemeral data that can be replayed from a log (so persistence isn’t a big concern)
  • Frequently-updated data: writing to RAM is *much* faster than writing to disk
  • When latency is extremely important
  • When the same data is queried regularly, the volume of data is low, and/or you have a large hardware budget

For low-throughput applications or applications that lend themselves to horizontal scaling, these features may not provide enough benefit to get excited about. But, for high-throughput apps where latency matters, the performance increase can be substantial.

Subconscious is open-sourced using an MIT license, so feel free to use it for your own projects!

Interested in working with these technologies (and more)? We’re hiring software engineers.


dsert - Dictionary Assertion Library


I spoke at a recent New York Python Meetup Group event hosted by Stack Overflow that focused on a library we open-sourced called dsert.  At Paxos, we use it for API testing, and you might want to as well. Using dsert we can test all the fields of a python dictionary, even when we don't know all of their values. This ensures that tests are as explicit as possible. You can watch the full video below, or on our Paxos YouTube page.


The Blockchain is Evolutionary not Revolutionary

Many people across the technology and financial industries are dubbing blockchains as the greatest innovation since the Internet. However, a blockchain is comprised of a bunch of technologies that are actually pretty old. The biggest surprise when it comes to Bitcoin may be that it wasn’t invented a decade earlier using dial-up internet!

Most blockchains use the six major technologies below. In this post, we will examine each technology and explain the role they play in a blockchain.

Technologies of a Blockchain

Asymmetric Encryption

The magic of asymmetric encryption is that signatures accomplish the following:

  • Prove that the signer had access to the private key
  • Do not reveal the private key
  • Are trivial to verify, yet impossible to forge/alter

Bitcoin uses the secp256k1 parameters of the Elliptical Curve Digital Signing Algorithm (ECDSA). ECDSA was invented in 1985 and became an ISO, ANSI, IEEE and FIPS standard in 1998-2000. The major advantage ECDSA has over RSA is that ECDSA uses much smaller keys and signatures to achieve the same level of security. In other words, Bitcoin would have been possible with RSA... which was invented in 1977!

One of the main use cases for asymmetric encryption is public-key encryption. For example, Alice can encrypt a message using Bob’s known public key and send it to Bob over an untrusted network so that Bob can then decrypt the message using his private key. This feature of public/private key cryptosystems is not really used in Bitcoin.

Hash Functions

Hash functions take input data of arbitrary size and deterministically map it to an output of fixed size (typically smaller than the input size) that resembles random data. A key property of a hash function is collision resistance.

You may have noticed that each transaction and block in Bitcoin is represented by 64 hexadecimal characters. That’s because these “IDs” are calculated deterministically by serializing the transaction/block contents into bytes and then hashing those bytes (twice) using SHA-256. The result is the transaction/block hash.

This provides a convenient integrity check on transactions/blocks. Just as an asymmetric signature cannot be altered by a dishonest actor, the contents of a transaction/block cannot be tampered with due to collision resistance. This provides a very useful guarantee to all participants that their version of history is the same as all other participants on the blockchain. If two sources share the same current block hash, then they know they share every single input/output in every previous transaction/block.

Another use of hash functions in Bitcoin is that public keys are hashed in order to determine a Bitcoin address. This is a defensive protection against the future invention of a quantum computer that could break ECDSA (learn more here).

Bitcoin transaction/block hashes and merkle trees use two rounds of SHA-256, while address derivation uses two rounds of SHA-256 and one round of RIPEMD-160. SHA-256 was first published in 2001 and RIPEMD-160 was first published in 1994. Hash functions have been around a lot longer.

Merkle Trees

In order to package transactions into blocks, Bitcoin uses a Merkle Tree. This data structure takes a list of transactions and combines them using a binary tree structure, where the root node is called the Merkle Root. The killer feature of the Merkle Tree is that proof a leaf was included in the Merkle Root takes O(log(n)) space. Using this technique makes it possible to run a very secure Simple Payment Verification (SPV) bitcoin wallet on your phone without storing 100+ GBs of blockchain data.

The Merkle Tree was patented in 1979 and is used in distributed file systems like IPFS, file-sharing systems like BitTorrent and NoSQL databases like Cassandra.

Key-Value Database

In order to prevent double-spends (a key feature of a blockchain), you have to be able to quickly perform the following two database operations:

  1. Lookup if a transaction has already been spent
  2. Insert a new valid transaction

While NoSQL databases have taken off in popularity in recent years (due primarily to their ability to handle extreme scale), it is important to remember that NoSQL databases actually pre-date relational databases.

Bitcoin originally launched with BerkeleyDB, which was released in 1994. dbm, the key-value store that inspired BerkeleyDB, was released in 1979.

P2P Communication Protocol

Having nodes communicate directly with one another (as opposed to using a trusted third-party) is unlike most applications we use on a daily basis. However, it isn’t new.

Napster’s 1999 release is probably the most commonly known P2P network, but USENET predates it by two decades.

The internet provides an excellent network for P2P protocols. Gossip Network Protocols have been used in many NoSQL Databases including Amazon Dynamo, Cassandra and Riak.

Proof of Work

Proof of work (PoW) is a clever application of hash functions. It works by calculating the hash of a message, along with many different nonces, until you find a resulting digest that meets a rare criteria. Since each hash is equally unlikely to meet that criteria, specifying a hard-to-meet criteria (perhaps a hash that starts with several leading 0s) is a way to prove that someone spent their CPU cycles. Also, a correct solution will be trivial to verify.

By using proof of work to achieve consensus, it becomes unfeasibly expensive to attack the Bitcoin blockchain. This is a cornerstone feature for a permissionless network built on anonymity and strong distrust of other participants.

Some people find PoW to be inefficient, since the Bitcoin network spends enormous computing resources doing work that has no other value.

PoW was first invented in 1993, but was made famous in 1997 by Adam Back’s Hashcash.


Evolution of Blockchain Technology

The fact that blockchains use old technologies should not be taken to mean that they represent solved problems or are easy to deploy. After all, it isn’t the technologies in a blockchain that make it useful, it’s the clever way in which they interact. Scale, privacy, security and fault tolerance are all hard problems in computer science. To make things more complicated, blockchains are typically used to handle mission-critical operations (such as settling financial transactions like we are doing at Paxos), where serious bugs may not be acceptable to end-users.

A modern blockchain must optimize tradeoffs while making as few sacrifices as possible, a difficult and delicate balancing act.


We are looking for talented engineers to help us revolutionize how assets move through the global financial system. Apply now to join our growing team in NYC!  


Blockchain: Separating Hype from Substance Part Two

Part one of this two-part series took an in-depth look at the Bitcoin blockchain and the properties that add value in a variety of digital transactions. In part two, we look at the key properties of a blockchain, how they apply to public and private blockchains and the value of private blockchains for trading assets beyond bitcoins.

It’s always easiest to think of a blockchain as a subset of a database. To begin, let’s take a look at the six core properties of a blockchain which allow it to operate without a central authority or third-party intermediary: 

1. Provably Consistent: No conflicting transactions within the database and there are rules as to what can go in the database. Those rules are respected and the consistency of the database can be checked and proven.

2. Append-only: Database can only have new data added, old data cannot change or be deleted.

3. Ownable: Certain data in the database can be owned and only the owner may operate on that data through use of proofs.

4. Highly Available: Data must be available to whoever needs to check its consistency with database rules.

5. Canonical: There is exactly one true database. If there are multiple versions, it’s easy to determine which one is actually the true one.

6. Practically Immutable: It costs an impractical amount of time and/or money to subvert any of the above properties.



Bitcoin: The First Database with No Central Authority

There have been many databases with at least some of these properties. However, no database combined all of these properties until the advent of Bitcoin in 2009, making it the first distributed database that did not need a central authority to operate. As we mentioned in part one, having no central authority is desirable for many reasons, including faster transaction speeds, increased privacy and reduced transaction friction.

Let’s look at how Bitcoin establishes each property:

Provably Consistent

Bitcoin has rules for its database called libconsensus and they are literally codified in C++ (note: it took Bitcoin developers several years to do this). This provides provable consistency as any node on the network can check that the entire blockchain database respects the rules.


The Bitcoin blockchain is a database where each block contains the fingerprint of a previous block. Thus, the database can only append entire blocks at a time. Further, each block’s digital fingerprint changes if anything in the block changes. This makes it very difficult to alter a previous block.


Most coins (and thus, data) in the Bitcoin blockchain require proof of a private key. That means spending from a previously unspent output (aka UTXO) requires some proof that you have permission to spend it. This is done through public key cryptography and more specifically through something called the Elliptical Curve Digital Signature Algorithm (ECDSA).

Highly Available

Since Bitcoin is a decentralized public network, anyone can connect to the network and download the entire blockchain (~100GB) to check for consistency. As of this writing, there are approximately 5,500 nodes from which the blockchain can be downloaded and checked. 


Bitcoin solves the double spending problem through the use of something called proof-of-work (POW). The main property of proof-of-work is that it requires a tremendous amount of computation to create, yet very little computation to validate. Each block in the Bitcoin blockchain requires proof-of-work and all the computing power in the world dedicated to doing this takes roughly 10 minutes to find. This means that to create an alternate version of the blockchain would cost the same amount of computing power.

Practically Immutable

The main way Bitcoin provides immutability is through proof-of-work. At least 51% of the global Bitcoin network’s computing power is needed in order to be able to subvert Bitcoin’s canonical or append-only properties.

Similarly, you would need to bring down all 5,500 nodes (some of which are very hard to locate) on the network today to subvert the high availability of the Bitcoin database. The code for consistency is published and widely available and to subvert that part, each of the 5,500 nodes would have to be changed in the same way. Again, this is an unfeasibly expensive prospect. Additionally, public key cryptography is known to be an extraordinarily difficult problem to solve and the only known way to subvert it would be with computational power greater than all computing power in the world for billions of years.


It’s important to remember that the absence of any of these properties would necessitate a central authority. If you can have conflicting transactions, transfer of value would be fraught with risk. If database history can be changed, again, transfer of value would be fraught with risk. If there was no ownership, there would be no transfer of value. If there was no public availability of the database, then no one would be able to trust the database or transact on it. Without canonicity, double-spending would be possible causing transfers of value to be fraught with risk. Finally making everything prohibitively expensive to alter gives the Bitcoin network strong security.


Applying Blockchain Properties to a Private Blockchain

In contrast to a public blockchain like Bitcoin, all participants are known in a private blockchain. There are onboarding processes for each node and the absence of anonymity adds security, trust and data integrity to the shared database. These characteristics make private blockchains the preferred option for applications within the financial services industry.  

Like public blockchains, private blockchains can also operate without a central authority. Let’s take a closer look at how private blockchains can satisfy each property:

Provably Consistent & Highly Available

Generally, databases are said to be provably consistent if they can be constantly audited. Distributing the database to each node in a private network would certainly make this possible. The only requirements would be distributing a set of rules that dictate what data can be added and that the data indeed conforms to those rules. Making the data available is a problem that’s been largely solved. Most websites, for example, host the same data in different data centers around the world to make their websites available even when a couple hosts fail. Similar technologies exist for databases, even without the gossip network and large number of nodes that bitcoin uses.

Append-only & Canonical

Making a database append-only requires some checkpoints, which Bitcoin does through blocks. Creating an append-only database is something that already exists with a class of database storage units called WORM (Write once, read many). That, of course, relies on hardware properties to ensure that nothing is altered afterwards. In a private networking context, we will need a tool that is very expensive to reverse-engineer or change. This can be done a number of ways, including a private proof-of-work algorithm, merge-mining with Bitcoin or even signatures of parties that would otherwise not cooperate. The act of figuring out what’s canonical is what we would call consensus modeling. At Paxos, as befitting our name, this is an area of research where we figure out the tradeoffs to various consensus models.

Ownable & Practically Immutable

Ownability and practical immutability can be achieved using various forms of public key cryptography. Counterfeiting is virtually impossible (very expensive) and gives us practical immutability if the known parties of a private network sign.


Value of Private Blockchains for Trading Assets

For assets that are traded and settled digitally on a private blockchain (stock, bonds, etc.), the absence of a central authority means no third-party intervention is required. There are certainly regulatory considerations to retrofit into our new model. However, absent regulatory concerns, we can trade and settle traditional assets on a private blockchain with a similar level of security, efficiency and cost-savings as moving bitcoins on a public blockchain.

For assets that are either physical like gold or in another system like US dollars, third-parties are required to act as depositories. However, unlike a central authority, they are merely guarantors of the underlying asset on the private blockchain should the asset be requested. As a result, such third-parties don’t have to take on counterparty risks, operating costs or even privacy liabilities associated with being a central authority.

Blockchains cut out risks and costs present when a central authority is part of the trade settlement process. Private blockchains can enable faster settlement speeds, reduce liabilities and counterparty risk and bolster security, helping entire financial systems operate more efficiently.


We are looking for talented engineers to help us revolutionize how assets move through the global financial system. Apply now to join our growing team in NYC! 


Python 3.5+ Async: An Easier Way to do Concurrency

I spoke at a recent New York Python Meetup Group event hosted by Stack Overflow that focused on Async Programming. My talk was about thirty minutes long and you can watch the full video below. The video is also available on our Paxos YouTube page.



Blockchain: Separating Hype from Substance

“Blockchain” has been a popular term in the financial industry over the last four years. Countless articles have been written about its benefits, how this innovative technology can change the way business is done and what it means for the future.

Given the abundance of information and commentary out there, how do you separate hype from substance? What’s the actual end benefit of blockchain technology and does it have any value beyond Bitcoin?

This two-part series cuts through the noise, deconstructing what a blockchain is at its most fundamental level and exploring the technology’s uses in the financial services industry. Part one examines the Bitcoin blockchain and identifies inherent blockchain properties that add value for data integrity, privacy and security. Part two (due out in February) will dig deeper into the benefits of blockchain technology independent of Bitcoin and what a blockchain-like database with systemic integrity can bring to a business.


Blockchain Anatomy 101

Blockchain is often introduced as the technology behind Bitcoin. Though true, this simplified explanation obscures the fact that blockchain technology is a rigorous system that creates certain desirable properties and has a variety of potential applications beyond Bitcoin.

At the most fundamental level, it’s easiest to think of a blockchain (also referred to as a distributed ledger) as a subset of a database, or a data store with special characteristics. In the case of Bitcoin, the basic unit used in the blockchain are transfers of value called transactions. Transactions in this context are similar to the common English usage of the term, that is, an asset (in this case bitcoins) being transferred from one owner to another.

These transactions almost always require signatures which add easy-to-prove authorization. This is done through the use of public-key cryptography, and in Bitcoin’s case ECDSA (Elliptical Curve Digital Signature Algorithm). Such signatures are near-impossible to forge as they require immense amounts of computation (think every computer in existence for millions of years) to create without the private key.

Transactions are grouped into what are called blocks. Blocks are the mechanism by which the data store can be checked for consistency. This is done through two mechanisms.

First, all blocks point to a previous block. This allows them to be well-ordered and is the inspiration for the term blockchain. Each block extends a chain of blocks with the obvious exception being the first block, which is called the genesis block. Having an ordering of blocks allows for an ordering of transactions. This is important for auditing and validation purposes.

Standard composition of a blockchain, from newest block to the genesis block

Standard composition of a blockchain, from newest block to the genesis block

Second, each block must not only be self-consistent (no conflicting transactions within the block) but also consistent with the network rules (no conflicting transactions with transactions in previous blocks in the chain). This allows anyone with access to the entire blockchain to check that the rules of the network are respected. In addition, any capable machine can check that each new transaction (and thus, each new block) is consistent with all the transactions that came before. In short, any machine on the network has the ability to check the integrity of the database with respect to the network rules.


Bitcoin: The First Decentralized Digital Payment System

The blockchain, combined with something called proof-of-work, gives Bitcoin an extremely desirable property. Because the complete record of every transaction is shared publicly, decentralized and well-ordered through blocks, nobody can cheat by spending bitcoins they don’t have (double spending) or creating bitcoins that don’t exist. There’s a whole network that enforces which blocks/transactions are valid without a central authority. Essentially, Bitcoin is a digital payment system that eliminates the need to trust a third-party intermediary to execute transactions.

Think about that for a moment. Most digital payment systems require you to trust third-parties like banks and credit card providers. Conversely, in-person methods of payment like cash, gold or barter do not require trusted third-parties but they are non-digital and cumbersome in nature. Bitcoin has combined the best characteristics from both worlds.

Eliminating third-party intermediaries from the transaction process, as Bitcoin does, is valuable for three key reasons. First, third-parties have to make money and traditionally, they do this by adding cost to the system. For example, credit card providers take up to 3% of the transaction as fees for their service. Second, using a third-party intermediary adds risk. This may not be apparent when using reputable companies like Visa or Paypal, but the payment space is littered with companies that have gone bankrupt, had accounts seized or hidden insolvency by cooking the books (see Flooz, E-gold or Pay by Touch). Finally, the involvement of third-parties reduces privacy. An In-person cash transaction is nobody’s business but the two people involved in the exchange. When a third-party enters the picture, they gain access to sensitive personal and financial information.

In addition to its strengths as a decentralized digital payment system, the Bitcoin blockchain is also valuable in that due to proof-of-work, the ledger is immutable (for all practical purposes). That means no one can go back and change numbers around to make things look different. This is great for auditing and ensuring systemic integrity.

Blockchain and the Global Financial System of the Future

It’s important to remember that third party intermediaries aren’t present only in consumer transactions like buying groceries with a credit card. Thousands of times every second, third-parties help execute the buying/selling of stocks, derivatives, currency, precious metals and other assets. In short, most transactions today require both a third-party intermediary and centralized infrastructure that adds cost, risk and privacy-loss to the transaction.

The question, then, isn’t about the utility of blockchains in financial transactions, but whether it is possible to use blockchain's properties to increase privacy while decreasing cost and risk when executing these transactions. When discussing the financial applications of blockchains, there is less interest in the technical specifics as they pertain to Bitcoin. Rather, the excitement is in how blockchain technology’s inherent desirable properties can be applied to modernize how assets (beyond bitcoin) move through the global financial system.

Part two, set for release in February, digs deeper into the benefits of blockchain technology (digital exchange of assets without a central authority, practical immutability) without the mechanisms of Bitcoin and what a blockchain-like database with systemic integrity can bring to a business.

Engineering at Paxos

At Paxos, we are using blockchain technology to build the next-generation infrastructure that will power capital markets for years to come. Our blockchain platform, Bankchain, streamlines and automates post-trade settlement, the process that underpins and serves as the foundation for the global financial system.

Although trading speeds, regulations and the complexity of assets have changed, settlement has not. Today’s post-trade settlement systems were designed decades ago, many during the Nixon administration. Disrupting deeply entrenched global financial operations and processes is a huge challenge. We need the best engineers to solve the complex problems facing today’s capital markets and help us to modernize how assets move through the global financial system.

Our engineers come from a variety of backgrounds ranging from top financial institutions to innovative startups. Many of our engineers have also worked on innovative projects within the Bitcoin and blockchain community before joining Paxos. Regardless of where they come from, we strive to hire the best and expect our engineers to maintain the highest standards.

Paxos Engineering Core Values

Our core values guide the engineering team’s work and drive our culture of pragmatic innovation. 

Be Pro-Active

BE PRO-ACTIVEPaxos engineers don’t wait around to be assigned tasks, they take the initiative. We would rather have our engineers ask for forgiveness than permission.

Be Accountable

BE ACCOUNTABLEOur engineers see their projects and responsibilities through to the end and are always accountable to one another.

Don’t Be Afraid of Failure

DON’T BE AFRAID OF FAILUREPaxos engineers are never afraid to experiment with new ideas. We don't consider churn a bad thing and our engineers are encouraged to refactor code and make it better. We expect that many of our ideas won’t work and that the ones that do will need a lot of iterations. We use every instance as an opportunity for improvement.

Be a Force Multiplier

BE A FORCE MULTIPLIER"10X engineer" is a common term at tech companies, but isn't in our vocabulary at Paxos. We expect our engineers to be Force Multipliers. We do the grunt work and automate processes to make each other more productive and effective.

Keep It Simple

KEEP IT SIMPLEWe believe in writing simple readable code and using the right tool (framework, language) for the job. We only choose technology because it fits a business need. 

Committed to Excellence

COMMITTED TO EXCELLENCEOur engineers do not accept “good enough” and understand that there are no shortcuts to success. We have high standards and do everything in our power to ensure that our products, behavior and thinking meet those standards every day.

Show Respect

SHOW RESPECTGood ideas can come from anywhere. At Paxos, no engineering idea goes unheard. We treat everyone equally and with the utmost respect. We believe that the best teams allow everyone equal time to convey their opinions and listen actively to those expressed by others.   

Since we began our journey of discovery with customers over a year ago, we have fielded a wide variety of questions about blockchain and distributed ledger technology (DLT). It has become increasingly evident to us that while there is a lot of excitement about blockchain technology, there is also a lack of clarity on its applications and potential for disruption.

As engineers and blockchain evangelists committed to openness, we look forward to educating the broader engineering community about this innovative technology. We are big proponents of open source and passionate about sharing our code, perspectives, challenges and learnings. The Paxos Engineering Blog is the first step in sharing our knowledge with the community.



Latest Posts Delivered to your Inbox