Smart Contract Verification
Last updated
Last updated
Smart Contract VerificationDetails about the verification microserviceA microservice written in Rust provides fast and efficient contract verification. The application runs as an HTTP server and verification requests are made using REST API.
Http API
In order to succesffully intall and configure the servie docker and docker compose are needed.
Service Configuration
Update Blockscout to the 4.1.8 release
Add the following ENV variables:
ENABLE_RUST_VERIFICATION_SERVICE=true
RUST_VERIFICATION_SERVICE_URL=... (verifier endpoint, see http configuration for more details)
If running locally, the smart-contract-verifier is integrated with docker-compose
The service consists of 2 parts, a verification library and a transport layer that serves requests.
Verification Library: Provides verification and a communication interface for verification functions. Includes modules for Solidity, Sourcify, and Vyper.
Transport Layer: Verification requests are sent via HTTP.
We've seen a 10X improvement in verification speed over the previous implementation. Benefits of the new microservice include:
Scalability: Remove CPU intensive tasks from the primary Blockscout instance. Modularity provides the ability to scale quickly.
Speed: Using native compilers greatly increases compilation speed relative to the JS wrapped versions running on Node.js used previously.
Improved efficiency. New algorithm supports contracts that contain several metadata hashes, improving efficiency.
On startup, the library must retrieve all available compilers. Compiler information consists of a link to download the compiler and its hashsum [example]. A separate thread also starts during initialization to update this list periodically.Library processes differ based on verification for Solidity/Vyper contracts or verification via Sourcify. Both are detailed below.
Verification processes are similar for contracts written in Solidity and Vyper.For Solidity contracts, verification is available for multiple files or standard json input. Single file verification is also possible - users still verify via the multiple file method and only upload a single file.For Vyper contracts, only multiple file verification is available.
Each request contains the following values:
Deployed bytecode: bytecode of the contract stored on chain
Creation transaction (tx) input: input data provided in transaction to create the contract
Compiler version: Version of compiler used to compile the contract
Content: Verification method related fields. E.g. for Solidity multiple files those fields are:
Sources: mapping from source files to their contents
Evm version: version of the EVM used to compile sources
Optimization runs: optional parameter that enables optimizations and specifies number of optimization runs
Contract libraries: optional parameter that specifies addresses of the contract libraries used in compiled contracts
1.Download the compiler using the specified compiler version from the url retrieved on initialization (or update).
2.Compilation: compile using the provided and slightly modified (see Verification Algorithm section) sources.
3.Verification: compare compiler output from step 2 with bytecodes (deployed bytecode and creation tx input) provided by the caller. During verification all contracts obtained through local compilation are checked one by one, and if any corresponds to provided bytecodes, the verification is considered successful. Contract paths and names are already indexed from available contracts, and constructor arguments are extracted.
4.Return Values: See below.
As a result of successful verification the following struct is returned:
Compiler input: Data used as an input for the compiler
Compiler version: version of the compiler used in verification (corresponds the the version provided in request)
File path: path of the file containing verified contract
Contract name: name of the verified contract
Abi: application binary interface of the verified contract (https://docs.soliditylang.org/en/v0.8.13/abi-spec.html?highlight=abi)
Constructor arguments: optionally returned constructor arguments used during on chain creation of the contract.
In case of an error, enum is returned with the corresponding error and its description (e.g. if compilation failed, a Compilation error is returned with the error data obtained from the compiler).
The request is sent via proxy to the Sourcify service. It accepts the following arguments:
1.Address of the contract to be verified.
2.Chain id where the contract is deployed.
3.Files required for Sourcify verification.
4.(optionally)Contract index, if the provided files contain multiple metadata files (i.e. multiple contracts).
These arguments are sent to the Sourcify Api, which returns the verification status.
Either the data provided was valid and the contract was verified by Sourcify, or the contract was previously verified. To obtain the actual data used for verification we send another request to retrieve source files used in verification. We expect data used for verification to be in the metadata.json file, so it is parsed and the information retrieved. This includes file and contract names, compiler and evm versions, info about optimizations, contract libraries, abi, and sources.Note: Constructor arguments cannot currently be obtained from the metadata via Sourcify. This requires the input used for contract creation transaction and the bytecode resulting from local compilation, which are not provided.
If verification fails we return an error with description obtained from Sourcify response
Verification is divided into 2 parts to effectively deal with metadata hash differences in contracts. Often, very similar contracts will have very different metadata hashes based on small changes within the contract (ie an extra space in the source code). To address this, the algorithm:
1.Extracts all metadata hash fields from creationTxInput.
2.Compares non-metadata sections of the onchain bytecode with the results of local compilation.
Metadata hashes are extracted by compiling the same contract twice (with a slight modification to the 2nd contract that does not change any other aspect of the resultant bytecode). The resulting creationTxInputs are compared and metadata hashes extracted by finding which parts of the inputs differ.
An additional space in the source code can be used for the modification process. In Example 1 and Example 2 the only difference between both contracts is a space in the beginning of Example 2. However, this still completely changes the metadata hash (metadata hash differences shown in bold red).Note: To update the metadata hash in production we use an additional library passed into the compiler. It does not effect the resultant bytecode, but the compilation input is altered, leading to updated metadata and an updated metadata hash.
// pragma solidity ^0.8.7; contract Main {}creationTxInput: 6080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfea26469706673582212203cc19b771ee83d8cc6995191bd9092a82f3e66b95aacaf1cfc893d3dc27abcd264736f6c63430008070033
// pragma solidity ^0.8.7; contract Main {}creationTxInput: 6080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfea2646970667358221220b3902f5797f05bb62858c8d41f09402093c85957c4ae1d773e4eb1cb2000f09364736f6c63430008070033
1.Compile the original contracts using data provided by the caller then compile those contracts again while passing in the additional library. The result is two slightly different CreationTxInputs if there is any encoded metadata.
2.Separate the MainPart and the MetadataPart.
1.MainPart consists of consecutive bytes which are not related to the metadata hash (all bytes are the same for both local creationTxInputs). MetadataPart consists of consecutive bytes related to metadata hash (some bytes differ between local creationTxInputs).
3.Iterate through the two local contracts byte by byte to find the first byte that differs.
1.If there are no such bytes, then there is no metadata hash and the whole creationTxInput consists of just one MainPart.
2.If any differences are found, identify it as a hash section of the metadata hash.
3.Iterate back byte by byte to find the beginning of that metadata hash. When we find the beginning we know where that metadata hash starts and ends. All non-categorized bytes before the start are identified as the MainPart and bytes between the start and end identified as the MetadataPart. Continue until there are no uncategorized bytes.
4.Once the entire local creationTxInput is categorized, we compare those parts with the corresponding remote creationTxInput bytes. For the MainPart remote creationTxInput must have exact equivalence. For the MetadataPart we compare some parts of the metadata (e.g. that encoded solc versions are the same), but in general we do not require them to be equal.
ExampleLocalCreationTxInput1: 608060405234801561001057600080fd5b506040518060200161002190610050565b6020820181038252601f19601f820116604052506000908051906020019061004a92919061005c565b5061015f565b605c806101ac83390190565b8280546100689061012e565b90600052602060002090601f01602090048101928261008a57600085556100d1565b82601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b5b808211156100fb5760008160009055506001016100e3565b5090565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052602260045260246000fd5b6000600282049050600182168061014657607f821691505b602082108103610159576101586100ff565b5b50919050565b603f8061016d6000396000f3fe6080604052600080fdfea2646970667358221220187a584947e37ebca43172929f7e159fe28696b0edd97bbfad5b0a265f6f886964736f6c634300080e00336080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfea2646970667358221220805aa9fe4ec055702024afa5ac21c2104f0b14be0ffab086f0d6e9b5701073f864736f6c634300080e0033Parts 1: [ MainPart(608060405234801561001057600080fd5b506040518060200161002190610050565b6020820181038252601f19601f820116604052506000908051906020019061004a92919061005c565b5061015f565b605c806101ac83390190565b8280546100689061012e565b90600052602060002090601f01602090048101928261008a57600085556100d1565b82601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b5b808211156100fb5760008160009055506001016100e3565b5090565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052602260045260246000fd5b6000600282049050600182168061014657607f821691505b602082108103610159576101586100ff565b5b50919050565b603f8061016d6000396000f3fe6080604052600080fdfe), MetadataPart(a2646970667358221220187a584947e37ebca43172929f7e159fe28696b0edd97bbfad5b0a265f6f886964736f6c634300080e0033), MainPart(6080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfe), MetadataPart(a2646970667358221220805aa9fe4ec055702024afa5ac21c2104f0b14be0ffab086f0d6e9b5701073f864736f6c634300080e0033) ]LocalCreationTxInput2: 608060405234801561001057600080fd5b506040518060200161002190610050565b6020820181038252601f19601f820116604052506000908051906020019061004a92919061005c565b5061015f565b605c806101ac83390190565b8280546100689061012e565b90600052602060002090601f01602090048101928261008a57600085556100d1565b82601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b5b808211156100fb5760008160009055506001016100e3565b5090565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052602260045260246000fd5b6000600282049050600182168061014657607f821691505b602082108103610159576101586100ff565b5b50919050565b603f8061016d6000396000f3fe6080604052600080fdfea26469706673582212202e82fb6222f966f0e56dc49cd1fb8a6b5eac9bdf74f62b8a5e9d8812901095d664736f6c634300080e00336080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfea2646970667358221220bd9f7fd5fb164e10dd86ccc9880d27a177e74ba873e6a9b97b6c4d7062b26ff064736f6c634300080e0033Parts 2:[ MainPart(608060405234801561001057600080fd5b506040518060200161002190610050565b6020820181038252601f19601f820116604052506000908051906020019061004a92919061005c565b5061015f565b605c806101ac83390190565b8280546100689061012e565b90600052602060002090601f01602090048101928261008a57600085556100d1565b82601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b5b808211156100fb5760008160009055506001016100e3565b5090565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052602260045260246000fd5b6000600282049050600182168061014657607f821691505b602082108103610159576101586100ff565b5b50919050565b603f8061016d6000396000f3fe6080604052600080fdfe),MetadataPart(a26469706673582212202e82fb6222f966f0e56dc49cd1fb8a6b5eac9bdf74f62b8a5e9d8812901095d664736f6c634300080e0033),MainPart(6080604052348015600f57600080fd5b50603f80601d6000396000f3fe6080604052600080fdfe),MetadataPart(a2646970667358221220bd9f7fd5fb164e10dd86ccc9880d27a177e74ba873e6a9b97b6c4d7062b26ff064736f6c634300080e0033)]
Constructor arguments are extracted at the end of the verification process. Constructor arguments are appended at the end of the creationTxInput returned by the compiler. So, to extract these arguments we remove parts presented in local creationTxInput, and consider everything else as likely constructor arguments.After that we attempt to map potential constructor args to the constructor abi of the contract. If successful, extracted args are returned as actual constructor arguments.
Currently, verification requests are sent via HTTP requests. This module runs an HTTP server and provides endpoints through which external services (main Blockscout instance, in our case) send verification requests and receive responses. Endpoints provided by the service and configuration details are described in the corresponding Readme file.Currently only synchronous endpoints are available. In the future a new service that handles asynchronous requests on top of the current smart-contract-verifier may be implemented.
