Exploradors

Expert Solidity Bootcamp – Week 1

I applied to the Expert Solidity Bootcamp by Encode this summer and, to my surprise and excitement, I got accepted in the program. The program consists in 1,5hour of class, 4 days/week during 5 weeks . There is material and homework everyday. Here I am going to report my progress in a note format.

Week1 Summary

During week 1 we saw a considerable amount of material and theoretical concepts. We went over 4 lessons and touched topics such as EVM, storage, Solidity and upgradeable patterns.

Since a base knowledge was already expected, we didn’t dive much into Solidity specifically, but we did saw some highlights on the new compiler updates and upgradeable patterns. The focus was more on the stack machine and how storage and memory worked.

I felt pretty comfortable with the concepts, as most of them were already very familiar, but it’s true that they lead me into some rabbit holes (described below), which will result in creating new separate pages, for tackling my research in those topics specifically. As a self goal, I have planned to implement some of the concepts we saw in a practical manner (see Proof of Concepts), to better integrate the knowledge.

Ethereum Overview

Modular Blockchains and the Data Availability Problem

As a first introduction of the content, we saw an overview of the different ecosystem existing layers. Besides the Execution layer and Consensus layer, it was mentioned Data Availability layer, with examples like zkPorter, Polygon Avail or Celestia. This encompasses the definition of Modular Blockchains (check this great article here).

Execution and Consensus layers are quite well defined, so I didn’t enter into expanding more there. However, the Data Availability distinction rang a bell, so I saw an opportunity for research in that direction.

It all starts with light nodes. Full nodes compute and verify each and every transaction, while light nodes only download block headers and assume the transactions are valid. They rely on fault proofs generated by the full nodes. This model is based on the honest minority assumption, relying on at least one honest full-node submitting fault proofs, and are used also by Optimistic Rollups (Arbitrum, Optimism, etc.).

However, a different scheme is used for Zero Knowledge Rollups, where the optimistic assumption is removed. For that, Validity Proofs are used, guaranteeing the atomicity of all state transitions, with a sequencer/prover model, where sequencers handle computations and provers generate the proofs, based on the zero knowledge primitives SNARKs and STARKs (zkSync, Starknet, Polygon zkEVM, etc.)

Consensus Mechanisms. Liveness vs Finality

We saw the recent change from PoW to PoS, as well as some of the consensus mechanisms. There are 2 views:

  • Favoring Finality: such protocols can halt in favor of guaranteeing security and finality of the sate, like Tendermint.
  • Favoring Liveness: such protocols never halt, but they may not come to finality, giving place to re-orgs, like Bitcoin.

One of the things that changed with the merge was the Consensus mechanism, by combining GHOST (favoring liveness properties) and Casper (favoring finality). In summary:

  • Epochs are in ~6,4 mins
  • Epochs have 32 slots
  • Validators attest to one slot (12k attestations per block) – within an epoch a validator will only vote once
  • A slot occurs every 12s
  • One validator is chosen to submit a block within that slot
  • If the validator does not submit a block, the slot is empty
  • The first block within an epoch is a checkpoint block
  • Finality is achieved when sufficient votes are reached, generally after 2 epochs

World State

The world state is a mapping between addresses (160 bit identifiers) and account states. An account state is formed by:

  • nonce: scalar value (number of tx’s or number of contract-creations made by that account if it’s a contract). Here I am wondering how will we keep track of the nonce when we work with account abstractions (added into the rabbit holes list)
  • balance: scalar value (wei owned by that address)
  • storageRoot: 256-bit hash of the root node of a Merkle Patricia tree
  • codeHash: the hash of the EVM code of this account. Unlike the other fields, it’s immutable. For EOA’s it’s an empty field

Transaction Types

A transaction is a cryptographically-signed instruction send by an EOA (cannot be a contract). There are 3 different transaction-types (2 in the notes + 1 that I am adding).

Legacy (type 0)

  • type
  • nonce
  • gasPrice: number of wei to be paid per unit of gas
  • gasLimit: maximum amount of gas that should be used in this transaction (paid upfront)
  • to: 160-bit address of the call’s recipient, or ∅ on contract creation transactions
  • value: number of wei to be paid
  • r, s
  • data: not in the notes, but I believe it’s indeed the case

EIP-2930 (type 1)

  • accessList: list of access entreid to warm up. Each access list entry is a tuple of an account address and a list of storage keys
  • chainId
  • yParity: in legacy tx this is combined with the chainId

EIP-1559 (type 2)

  • maxPriorityFeePerGas: the maximum fee per gas they are willing to give to miners to incentivize them to include their transaction
  • maxFeePerGas: the maximum fee per gas the tx is willing to pay total, which covers both the priority fee and the block’s network fee per gas
  • gasLimit
  • gasPrice: removed! previous to this change, it represented both the wei paid by the signer per gas for a transaction as well as the wei received by the miner per gas. Now only represents the amount of wei paid by the signer per gas, and the amount a miner was paid for the transaction is no longer accessible directly in the EVM.

Transaction Receipt

It’s a tuple consisting in 5 elements:

  • tx type
  • status code of the tx
  • cumulative gas used in the block
  • set of logs created through the execution of the tx
  • bloom filter composed from information in those logs

Bloom Filters

Bloom filters are a probabilistic way of checking the membership of an element within a set. The answers are:

  • miss == definitely not present
  • hit == probably present

They are the inverse from caches, where:

  • miss == probably not present
  • hit == definitely present

I am not going to expand more here, as this is another great candidate for making a POC (see below).

Ethereum Roadmap

We saw a couple of EIPs that have been or will be implemented. I’m mentioning them here as they offer a good chance for exploring more in depth in the future, but I will create separate entries going in more detail.

EIP-2929 Gas cost increases for state access opcodes

Historically, storage-accessing opcodes have been underpriced. This allowed for DoS attacks (see the Shangai attack from 2016), where the attacked was sending transactions that access or call a large number of accounts.

This EIP is already in prod and does the following:

Increases gas cost for SLOAD, *CALL, BALANCE, EXT* and SELFDESTRUCT when used for the first time in a transaction.

EIP-2929 spec

Gas costs were increased to mitigate this, but after some time, it was seen that they were not increased enough. So, this EIP increased the costs of these opcodes by a factor of ~3, reducing the worst-case processing time to ~7-27 seconds.

EIP-2930 Optional Access List

As a result of the previous EIP, the Optional Access List improvement was proposed to mitigate some of this costs for legit use-cases. This EIP is also already in prod and does the following:

Adds a transaction type which contains an access list, a list of addresses and storage keys that the transaction plans to access. Accesses outside the list are possible, but become more expensive.

EIP-2930 spec

I haven’t used this myself nor heard examples where the access list was used, so this is a great candidate for being my first Proof of Concept (see below).

EIP-4337 Account Abstraction

This represents the shift from EOAs into smart contract accounts (not implemented yet, at today’s date). Again, I am not going to expand here, as this deserves going down the rabbit hole and a separate static page.

Cryptography Review

We didn’t go into detail, but listing here a couple of concepts:

  • Elliptic Curves Digital Signature Algorithm (ECDSA): only a signature algorithm – cannot be used for encryption unlike RSA or AES
  • Signing using ECDSA: signatures consist of 2 integers (r and s). Ethrereum also uses a recovery identifier (v)
  • Signature: {r, s, v}

The simplified signing process is (taken as-is from the Encode material):

  1. Calculate a hash ( e ) from the message to sign.
  2. Generate a secure random value for k .
  3. Calculate point (x₁, y₁) on the elliptic curve by multiplying k with the G
    constant of the elliptic curve.
  4. Calculate r = x₁ mod n . If r equals zero, go back to step 2.
  5. Calculate s = k⁻¹(e + rdₐ) mod n . If s equals zero, go back to step 2.

The v value is used to protect from replay attacks and it’s calculated as:

v = {0, 1} + chainId*2 +35

Rabbit Holes

Here are the derived topics I would like to investigate further, and generate some separate documentation:

  • Polynomial Commitment Schemes: Data Availability Sampling (DAS) seems to be one of the big challenges for L2’s and rollups. In fact, this is one of the motivations of EIP-4844 about Proto Danksharding. One of my side investigation for this topic is Polynomial Commitment Schemes and KZG.
  • SNARKs and STARKs: validity proofs and sequencer/prover model
  • Combining GHOST and Casper: article, slots, epochs etc.
  • Modified Merkle Patricia tree: for matching addresses and account states
  • Nonce with EIP-4337: how will we keep track of the nonce when we work with Account Abstractions?
  • Storage Root: how is the storage root used
  • Bloom Filters: how are bloom filters used
  • EIP-4337 Account Abstraction

Proof of Concepts

Here are my ideas on implementing practically different proof of concepts, in order to comprehend deeply some of the concepts we saw in the course:

Solidity

In the second lesson we saw an overview of Solidity. We didn’t emphasize it much as Solidity knowledge was expected, so here I am only going to highlight some of the important things I found useful to remember.

Function Visibility

VisibilityContract itselfDerived ContractsExternal ContractsExternal Addresses
public✔︎✔︎✔︎✔︎
private✔︎
internal✔︎✔︎
external✔︎✔︎

Note that adding the public keyword to a variable will generate a getter function for that variable.

  • view: can only read from the contract storage (i.e. getter functions)
  • pure: can neither read nor modify the contract storage (i.e. used for computation, like mathematical operations)

Events

  • Adding indexed keyword will enable to search for events via web3 in the frontend.
  • Events are only visible outside the contract.

Mappings

  • Mappings do not have a length
  • Every possible key exists and is mapped to a value, which is the default one for the data type

Memory, Storage and Calldata

  • Storage: it’s permanent, can be accessed across all functions and forms part of the contract’s state
  • Memory: temporary, only accessible within a function and discarded after execution is completed.
  • Calldata: location where external values from outside a function into a function are stored. Are non-modifiable and non-persistent
function test1 (string memory _exampleString) ... {
   _exampleString = "example" // memory can be modified
}

function test2 (string calldata _exampleString) ... {
   _exampleString = "example" // calldata cannot be modified
}

Constant and Immutable variables

They both cannot be modified after the contract has been constructed. There is a subtle difference:

  • Constant variables: can never be changed after compilation. They can be defined at the file level
  • Immutable variables: can be set within the constructor

Interfaces

They specify the function signatures, but the implementation is done in child contracts.

interface Example {
    function getValue(address contract) external returns (uint value);
}

Errors

  • require: either creates an error without any data or an error of type Error(string). Used to ensure valid conditions that cannot be detected until execution time. I.e. conditions on inputs or return values from calls to external contracts
  • assert: creates an error of type Panic(uint256). Used to test internal errors and to check invariants. Functioning code should never create a Panic
  • revert: like a throw statement, causes the EVM to revert. Usually used require instead
  • try/catch: used to catch errors in calls to external contracts
  • Custom Errors: they are gas efficient and readable
    • They cannnot be overloaded or everridden but can be inherited
    • Instances of errors can only be created using revert
error NotOwner(uint address)
...
revert NotOwner(address)

Accessing External Contracts and Libraries

  • Compile time: if your contract uses another contract or library (inheritance or external function call) , the compiler needs to have the code available. We use import / or copy the code in your file (flattening). The inherited code is merged into your contract and becomes part of your bytecode.
  • Runtime: if your contract make calls to other contract’s functions (deployed on a different address), we need to have the function signature available (checked at compile time) and the contract’s address, so we define an interface with the function(s) we want to use, so signatures are derived
interface Uniswap {
   function swap(uint256,address)
}
contract MyContract {
   Uniswap uni;
   constructor(Uniswap _uni) {
      uni = _uni;
   }
}
  • Pre-compiled contracts: they do not execute inside the smart contract, but part of the client specification. The existing 9 pre-compiled contracts live in the addresses 0x01 …0x09. They are used for:
    • Elliptic curve digital signature recovery
    • Hash methods to interact with bitcoin and zcash
    • Memory copying
    • Methods to enable elliptic curve math for zero knowledge proofs

Wormholes

  • Variables at file level: when do we use constants at the file level? Which is their visibility? Are they used for gas optimization purposes?
  • Return values in Interfaces: why we should add return, if function selector does not need it for calculating the signature? Bug related to that topic

Matter

  • Flattening: research on flattening plugins (Remix/Truffle) and try it
  • Return values: try contract with no return value in the interface (see rabbit hole above)
  • Pre-compiled contracts
  • Gas refund: check different gas refunds for deleting values. See Solidity Yellow Paper: “a gas refund of 15,000 is given for each state slot that is set to zero. These refunds are calculated during execution but only paid back post-execution.“

EVM

Ethereum State

There are 3 tries:

  • World State
  • Transaction
  • Transaction Receipt

View Functions

  • They don’t modify the state.
  • The opcode used is STATICCALL
  • For library, view functions don’ t have run-time checks that prevent state modifications

An interesting thing to note is that emitting events does modify the state, as well as calling any function not marked view or pure.

The EVM

From the Ethereum docs, the EVM is a stack machine with a 1024 items depth. Each item is a 256-bit word. This length was chosen to facilitate the use of  Keccak-256 hashes or secp256k1 signatures. The first 16 elements in the stack top can be manipulated/accessed at once (or stack too deep error).

  • Memory: during execution, the EVM maintains a transient memory (as a word-addressed byte array), which does not persist between transactions.
  • Storage: part of the global state. Contracts, contain a Merkle Patricia storage trie (as a word-addressable word array), associated with the account in question and part of the global state.

There is the concept of Call Stack which indicates an array of addresses of contracts being called, added or removed from the call stack as the CALL opcodes are executed.

Memory

Memory is a byte-array, starts with zero size but can be expanded in 32-byte chunks (256-bit), by accessing/storing memory at indices greater than its current size. Opcodes:

  • MLOAD: loads a word from memory to stack
  • MSTORE: saves a word to memory
  • MSTORE8: saves a byte to memory

Memory expansion costs scale linearly for the first 724 bytes and quadratically after that.

The Free Memory Pointer keeps track of where the queue ends and free memory starts, so more data can be saved. Once new data comes in, the free memory pointer is updated to reflect the first free slot at the end of the queue.

Storage

  • Fixed size variables: data is sotred contiguosly
  • Variable length items: the storage contains a pointer to another area of the storage where the variable starts

Sol2uml is a great tool that helps you visualize the storage layout for any contract.

Rabbit Holes

  • Recursive Length-Prefix Serialization: standardizes the transfer of data between nodes. See
  • Transaction processing: transaction validity checks
  • Opcodes: check EVM opcodes

Proof of Concepts

  • Grafana + Geth: setup Grafana with geth local node. See
  • Stack Machine: build a simplified 256-bit word state machine
  • Storage layout: checkk storage layouts using sol2uml tool

Upgradeability

We saw different patterns and the most common used at the moment

  • Transparent Proxy
  • Universal Upgradeable Proxy Standard (UUPS) – cheaper than Transparent proxy
  • Diamond Pattern

Storage layout

  • State variables are stored in a compact way, contiguously
  • Dynamic sized arrays and mapping are considered to occupy 32 bytes and contain a pointer

The correct approach is to only append to the storage when we upgrade.

We don’t use constructors rather an initializer function.

Metamorphic Contracts

To preserve the contract address among upgrades, but no tits state. It relies on the contract calling selfdestruct and then being re-deployed using CREATE2.

Wormholes

  • Unstructured storage: to prevent overwrite of the implementation variable. I.e. Open Zeppelin puts the implementation address at a random address in storage

Matter

  • Implement a Transparent Proxy
  • Implement a Universal Upgradeable Proxy
  • Try upgrade plugin

Leave a comment