The Ethereum cryptocurrency network experienced a massive outage on Wednesday. Initially it was assumed that it was caused by Infura – the servers supported by the Ethereum studio ConsenSys for the functioning of most applications in the field of decentralized finance (DeFi) and other services..
Infura stopped working around 11:00 Moscow time, along with which the functionality of popular applications such as Metamask, MakerDAO, Uniswap, Compound and MyCrypto was broken. Binance discovered conflicting transactions shortly thereafter and stopped withdrawing funds.
It gradually became clear that the real reason lay in the Go Ethereum (Geth) client bug, used by 80% of applications, which caused the network to split into two separate chains. The presence of differing transaction histories suggests that users interacted with different versions of the Ethereum blockchain for some time..
Later, Infura and others updated their clients, which allowed most of the services to be restored, but it took several hours to find out the reasons and transfer the infrastructure to the current software. All of them were using old versions of Geth and were separated from the users of the new client as a result of an “unannounced hard fork”.
Against the background of the incident, users began to ask questions about the decentralization of Ethereum and why the Geth developers did not cooperate with other teams to smoothly go through the update. Nikita Zhavoronkov, developer of Blockchair blockchain observer Nikita Zhavoronkov, considers “consensus failure” Wednesday as the most serious incident in Ethereum since the DAO fork in 2016.
How it was
Ethereum lead developer Peter Siladji released a report on the incident, confirming that the hard fork “was launched deliberately.” Ethereum scaling project Optimism announced that his team is behind the hard fork.
“We looked at Etherscan and saw that almost all of the nodes were upgraded to a fix-supported version. Then we decided to test the bug. Now it’s clear that it was stupid. We were not aware of the impact of the small number of nodes that were not updated, “said Optimism.
A hard fork is an update that is incompatible with an earlier version of the blockchain. Thus, the activation of the latest update on the Ethereum network naturally led to the emergence of two chains. One contained users and services that installed an updated version of Geth, the other did not..
“The fix was implemented several months ago, and only today the transaction that caused the split was carried out,” Zhavoronkov said in a conversation with CoinDesk.
Blockchair believed that the new version of Geth software contained a “small code change” and did not install it, as it would require stopping the service.
“The Geth team did change the implementation of the consensus in the v1.9.17 release, however the team did not create any new rules that the Ethereum community did not know or agree with,” Siladji wrote, adding that the rules were set in the improvement proposal Ethereum (EIP) three years ago.
It is noteworthy that the hard fork was originally conceived to solve the problem in consensus, which it caused. Developer John Young, who recently joined the Ethereum ecosystem, has been rewarded for discovering this and another vulnerability. Geth developer Martin Holst Svende announced that they did not alert the community to the change, as they did not want to draw attention to the vulnerability..
“Every major project should have contacts for security notifications to help keep updates running smoothly. When hard forks happen without warning, everyone who creates on Ethereum, including us, can lose money, “said Thesis founder Matt Luongo..
Thesis is behind the development of the tokenized bitcoin tBTC. Luongo said that the fork put the assets of tBTC users at risk, but not because of the chain split, but because of the inoperability of the service, as a result of which token holders could unwittingly violate their financial obligations.
Despite the problems caused by the fork, the price of ether increased by 4.6% after the news appeared, indicating that traders do not see this as a threat to the long-term well-being of Ethereum..
Zhavoronkov states that the Geth developers themselves did not expect such a development of the situation and, if they could have foreseen a large-scale split, they would certainly have announced a hard fork. Luongo shares this sentiment, describing the Geth team as “good developers lacking in infrastructure maintenance and funding experience.”.
Zhavoronkov also noted that the developers had no hostile intentions. “If Geth knew that this could happen, they would prepare a manual for node operators,” he added..
Community members continue to wonder why even Infura, a key element of the Ethereum ecosystem, was not aware of the bug before activating the fork to fix it..
“It’s kind of a gray area. An individual discussion is required in each case, – wrote Siladi. “We agree that transparency is essential and should be pursued as much as possible, but it’s also important to get the details right before heads roll. In the case of Ethereum, it takes a long time (weeks, months) for the node operators to update even to the planned hard fork. By emphasizing that the release contains important consensus or DoS fixes, we risk attracting attackers who will try to disable the network. “.
Ultimately, the Geth team decided that disclosing the vulnerability would be too much of a risk and kept it under wraps. “We argue that it worked, although the update took an unexpected turn during yesterday’s network split,” Siladhyi added..
Developers now have to learn a lesson from what has happened with regards to customer coordination amid the approaching release of Ethereum 2.0.
“The most important thing is that the guys are transparent, admit mistakes and grow,” said Luongo. – Monero has done a good job of dealing with consensus bugs in the past, as have Bitcoin and Zcash. There are many examples. Coordinating an entire industry is never easy, but avoiding coordination is extremely dangerous. Hopefully this fork will lead to a stronger relationship and a rethinking of how Ethereum projects interact with customer developers. “.
- Unfreezing of Parity Users’ Funds May Require a New Ethereum Hard Fork
Unfreezing of Parity Users’ Funds May Require a New Ethereum Hard Fork Against the background of the discovery of one of the largest vulnerabilities in…
- Istanbul hard fork scheduled for December 4th, next target is Ethereum 2.0
Istanbul hard fork scheduled for December 4th, next target is Ethereum 2.0 Ethereum startup PegaSys product manager Tim Beiko this Friday revealed an…
- Byzantium hard fork is less than a week away
Less than a week before the Byzantium hard fork. Is everything ready? Less than a week left before the next major update of Ethereum, the second largest…
- Ethereum threatens to split due to funds blocked on Parity wallet
Ethereum Threatened With Funds Blocked On Parity Wallet Ethereum faced an important choice regarding the technical vector of its network development,…
- Exchanges Limit ETH Withdrawal Amid Infura Malfunctions
Exchanges Limit ETH Withdrawal Amid Infura Malfunctions Infura’s infrastructure solution in the Ethereum ecosystem turned out to be unavailable this…