Tag Archives for " case study "

[Updated — The ENS Relaunch was a Success!] How a Project Manager Can Help the Ethereum Name Service Launch Succeed This Time

Published March 23, 2017 in Technical Project Management - 1 Comment
Timeline from Nick Johnson’s Postmortem

How a Project Manager Can Help the ENS Launch Succeed This Time

The first attempt to Launch the Ethereum Name Service (ENS) failed. Here’s how a project manager can make the next try successful.

The Ethernet Name Service (ENS) launched for a short time on March 14, 2017. The ENS community discovered two major security flaws shortly after launch. The team and community behind ENS decided to roll back the launch and try again.

Updated — I joined Nick Johnson and Alex Van de Sande on the ENS team shortly after the first launch attempt, to help with the relaunch. The relaunch was a success!

You can find the latest ENS news here.

I give the team a lot of credit for — 

  • Trying to launch the service in the first place
  • Getting the project to launch
  • Having the awareness to acknowledge problems and address them
  • Making the hard decision to pull back and try again
  • Their transparency in acknowledging what didn’t work and why
  • Their commitment to getting it right, by trying again

You can read about how things unfolded and security flaw details in -

Like Nick’s approach, I hope this post identifies places where a blockchain project manager can help projects like this succeed. It’s not intended to criticize, point fingers or place blame, AT ALL!

It’s also my offer to donate some of my time to provide project management oversight for the project!

(Nick Johnson, if you read this, let me know if you’d like to take me up on the offer.)

How a Blockchain Project Manager Help the ENS Launch Succeed this Time

I’ve written a post about the value project managers can contribute to blockchain projects. This post is a case study to that earlier thesis about the need for blockchain project managers to manage blockchain projects.

This rest of this post -

1 — Lists the causes and issues mentioned by Johnson and Maurelian

Nick Johnson and Maurelian identified many of the same causes and issues in their post-mortems. Each also identified other issues. This post focuses on the issues mentioned by both of them, for the sake of brevity.

2 — Describes how a blockchain project manager could have helped prevent each cause and issue.

Spoiler Alert — The Takeaway

Technical people develop technical solutions to difficult problems. That’s what they do. That’s what they want to do.

What’s missing is someone keeping an eye on the big picture to -

  • Identify issues before they become problems
  • Coordinate resources, making sure there are enough and guiding them to where they can be most effective
  • Gracefully slow down the project, only long enough to help everyone pick up their heads, adjust the end goal as needed and stay synchronized toward the agreed-to end goal
  • Do the work developers don’t want to do, e.g. document the project, coordinate audits, confirm thoroughness of plans, develop contingency plans etc

A blockchain project manager keeps an eye on the big picture. Keep reading for more detail.

(You can also contact me to discuss how I can make your blockchain project a success.)

Causes and Issues

Issue 1 — Incomplete Test Coverage

Johnson — Although the registrar had a suite of unit tests, this testing was not comprehensive. The unit tests were written after the contract by a different author. No concerted effort was made to ensure that all code paths or use cases were thoroughly covered.

Maurelian — Both of the issues uncovered could have been caught with tests to verify proper behaviour of the registrar. The behaviours which the issues enabled were not edge cases, but rather specifically prohibited…only 13…tests were on the Registrar contract, which has at least as many methods and many more requirements than that.

A blockchain project manager could(’ve) help(ed) by making sure the unit tests covered all code paths or use-cases by -

  • Developing a checklist of code paths and use-cases
  • Coordinating resources to execute the unit tests
  • Consolidating the results
  • Coordinate issue resolution

Issue 2 — No Formal and Independent Audit

Johnson — No independent, comprehensive audit was done.

Maurelian — There was no formal audit. There is unfortunately no single document summarizing what review was done, and what was found.

A blockchain project manager could(’ve) help(ed) by — 

  • Running the process to select an independent third party to conduct a formal audit
  • Acting as the liaison between the development and audit teams
  • Developing the review summary document

Issue 3 — Insufficient Code Review

Johnson -The changes to the contract that introduced the bugs were subject to review in a Pull Request, but the review did not catch the bugs. In future, more care is needed when reviewing changes to critical code to ensure reversions aren’t accidentally introduced.

Maurelian — (Referred to this issue as thrashing.) It’s nearly impossible to audit an evolving code base…Many changes were made between the Ropsten launch, and the live net launch…I actually looked at the commit that introduced these issues, but it’s a large contract…the effort required to…understand the full implications of the change felt onerous.

A blockchain project manager could(’ve) help(ed) by maintaining documentation that would make it -

  • Faster and easier to identify changes between commits.

Which would -

  • Make change implications feel less onerous to developers, who are already time-limited, allowing them to be more comfortable conducting code reviews.

A blockchain project manager could(’ve) also help(ed) coordinate more resources to conduct code reviews.

Issue 4 — No Official Bug Bounty

Johnson — A good bug bounty has…real value up for grabs…this is hard to do on a test net, where the tokens have no value, and stuff is expected not to work…WeiFund’s live net bug bounty, which held over 200,000 USD worth of real value in deployed contract, is a good counter-example. Blockchains present an interesting opportunity for a “natural bug bounty”, we as contract developers should be taking advantage of this.

Maurelian — No clear indication was provided to the community as to whether ENS was covered by the Ethereum Foundation’s bug bounty process. Had it been announced that ENS was covered, more attention would likely have been directed towards the code at an earlier stage in the process, resulting in the bugs being found at a stage where they could be easily remedied without affecting the launch.

A blockchain project manager could’ve helped by coordinating — 

  • Development of a Wei-Fund-like bug-bounty.
  • Alternative motivational programs, to encourage others to take part in the bug-bounty process.
  • Communicating a bug-bounty marketing plan.

Issue 5 — Incident response planning

(This one was only mentioned by Maurelian. I thought it was worth a mention, since a blockchain project manager could play a very valuable role here.)

Maurelian — …but bugs were always a possibility, so the need to delay should always be considered in advance. A common approach is to have a play book, basically, “if X happens, we will do Y”. I think that the reaction so far has been suitably transparent and proactive, but would have been better organized if the decisions had been made beforehand.

A blockchain project manager could(’ve) help(ed) by — 

  • Developing the play book.
  • Managing responses during implementation, according to the agreed-to playbook guidelines.

Here’s the Takeaway — Again!

I figured a repeat of the take away’s in order, if you read this entire post. And thanks for reading if you did!

Technical people develop technical solutions to difficult problems. That’s what they do. That’s what they want to do.

What’s missing is someone keeping an eye on the big picture to -

  • Identify issues before they become problems
  • Coordinate resources, making sure there are enough and guiding them to where they can be most effective
  • Gracefully slow down the project, only long enough to help everyone pick up their heads, adjust the end goal as needed and stay synchronized toward the agreed-to end goal
  • Do the work developers don’t want to do, e.g. document the project, coordinate audits, confirm thoroughness of plans, develop contingency plans etc

A blockchain project manager keeps an eye on the big picture.

P.S. — Working on a blockchain project? Contact me to discuss how I can help make your blockchain project a success.