Report for Himalia v0.1 & v0.2
5 weeks ago，Patract Hub applied a treasury proposal #61 for Himalia v0.1&v0.2, Now, we have finished all the work and you can review our codebase at：
- Himalia v0.1 PatractGo https://github.com/patractlabs/go-patract
- Himalia v0.2 PatractPy https://github.com/patractlabs/py-patract
PatractGo mainly completes the contract-related interfaces and APIs, and supports scanning and monitoring (observer) of events generated by the contract. PatractGo is based on Centrifuge's GSRPC. In the development process, in order to make the API more convenient to use and embed other services, we encapsulated GSRPC, providing API call based on context style. In addition, some interfaces that are not provided by GSRPC but are more useful for development and debugging have been added, such as encode and decode of AccountID in ss58 format, and unit test environment support.
PatractGo further encapsulates common ERC20 contracts, including API and monitoring support. This is convenient for processing ERC20 contracts, and can be used as an example to guide the development of other contract APIs. In the future, PatractGo will develop a code generator that automatically generates the corresponding go code based on the metadata of the contract. Since the logic of this generation process is the same for go and java, this part of the function is postponed to complete with PatractJ. An example of ERC20 is provided in the current version to show how to make packaging calls for a specific contract. For more information about PatractGo, please refer to the following function description and sample code in the project.
PatractGo also provieds the following founctions:
- dump provides an example of the ERC20 contract status synchronization tool. Dump will scan all events of the specified ERC20 contract. The number of tokens corresponding to the account is synchronized to a local database, and a simple query function is provided. This example can be used as an example of Oberver, and at the same time, it can be used as a tool to analyze the Token distribution of a specified ERC20 contract.
- rest use [github.com/patractlabs/go-patract/rest](https://github.com /patractlabs/go-patract/tree/master/rest) module. rest builds the url corresponding to the message based on metadata. Users can use this service to build the unsigned Extrinsic binary data of the specified contract, which can be sent to the trusted environment Third-party tool for signing. Based on this model, when contract developers use the Golang to develop contract back-end services, they can put the private key in an independent network environment and development environment to protect the security of the private key. In some scenarios, this processing is necessary , such as the account management of an exchange. As a developer-oriented SDK, programs containing
PatractGoand other SDK codes may not be able to run in some sensitive environments. Therefore, rest provides an HTTP-based service interface, which is also convenient for developers to access existing authorization system.
PatractPy mainly completes the support of the unit test environment and scans and monitors the events generated by the contract. PatractPy is based on Polkascan's Python Substrate Interface. In the v0.11.8 version of py-substrate-interface, they have provided good support for contract API, so PatractPy mainly supplements some of the missing functions of py-substrate-interface, such as subscription API and deserialization support for contract events. PatractPy can automatically construct a python contract object based on the message part of the metadata, and has a method to call the contract, similar to the implementation of api-contracts in polkadot.js. Therefore, after loading metadata, PatractPy can simply call the contract according to the definition of the contract's own method, without the need for static languages such as Java or Golang to provide a method of interacting with the contract through code generation. On the other hand, PatractPy completes the scanning and monitoring of events generated by the contract, and its implementation can also be used to process events from other modules.
As the most important function of PatractPy, we hope that it can support our Europa and complete unit testing for contracts based on it. On the one hand, as the realization of the test case of the PatractPy project itself, on the other hand, we can use python to implement the test case code for the contract based on the PatractPy to test and reproduce the test cases in complex scenarios, plus complete Python ecological test support can make smart contract development more convenient and efficient.
In the future, PatractGo and PatractPy will be integrated into our Redspot in the form of plug-ins. Therefore, for a contract development project, you can use Redspot to build a contract development framework, and then use Redspot's own polkadot.js or use a plug-inPatractPy to quickly and easily perform contract debugging and integration testing in our Europa, and then use PatractGo to call the contract. After the entire system has been debugged in the test environment, use the backend required by PatractGo to develop the contract and use it in the production environment. The next version of PatractJ and PatractN will be integrated into the Redspot system in the same way.
1. Himalia's future development plan#
M1: v0.1 PatractGo for GolangM2: v0.2 PatractPy for Python- M3: v0.3 PatractJ for JVM
- M4: v0.4 PatractN for .NET
2. PatractGo Report#
Based on PatractGo，we can interact with the contracts very easily。The design goal of PatractGo is closer to that the contract owner build its own business logic to interact with the contract on the chain through PatractGo, which is not suitable for the scenario of contract debugging. If you need to debug the contract, it is recommended to use ParactPy or polkadot.js for quick testing.
For PatractGo, we split the functions into three components:
- Contracts：provides the functions to interact with contracts
- Rest：provides a way to generate an offline signature for contracts.
- Observer: listens contract events and parses events by contract metadata.
2.1 API for Contracts#
By Native Contract API，We Can Call Contracts Runtime Earlier。
Put Contracts Code to chain：
// read the code wasm from file codeBytes, err := ioutil.ReadFile("/path/to/contracts.wasm") if err != nil { return err }
// create the api cApi, err := rpc.NewContractAPI(env.URL()) // read the abi(metadata) for contract metaBz, err := ioutil.ReadFile("/path/to/contracts_metadata.json") cApi.WithMetaData(metaBz)
// create context with from auth, like Alice ctx := api.NewCtx(context.Background()).WithFrom(authKey)
// put code _, err = cApi.Native().PutCode(ctx, codeBytes) // do next stepsGet Code from chain：
codeHash := readCodeHash() // get code hash
var codeBz []byte if err := cApi.Native().Cli.GetStorageLatest(&codeBz, "Contracts", "PristineCode", []byte(codeHash), nil); err != nil { return err } // codeBz is now codeInstantiate Contract in chain:
var endowment uint64 = 1000000000000
// Instantiate _, contractAccount, err := cApi.Instantiate(ctx, types.NewCompactBalance(endowment), types.NewCompactGas(test.DefaultGas), contracts.CodeHashERC20, types.NewU128(totalSupply), )api will return contractAccount, which can use it to call the contract.
For a contract, we can read or exec messages. Currently CallToRead and CallToExec implement a more basic level of encapsulation for contract calls, so contract developers need to write corresponding packaging functions according to the contract method, such as the ERC20 contract under the directory of PatractGo/contracts/erc20 example.
Read the total_supply of ERC20 contract, no request params:
var res types.U128
err := a.CallToRead(ctx, &res, a.ContractAccountID, []string{"total_supply"}, )Read the balance_of of AccountID for ERC20 contract:
req := struct { Address types.AccountID }{ Address: owner, }
var res types.U128
err := a.CallToRead(ctx, &res, ContractAccountIDForERC20, []string{"balance_of"}, req, )Call Transfer:
toParam := struct { Address AccountID }{ Address: to, }
valueParam := struct { Value U128 }{ Value: amt, }
return a.CallToExec(ctx, a.ContractAccountID, types.NewCompactBalance(0), types.NewCompactGas(test.DefaultGas), []string{"transfer"}, toParam, valueParam, )These behaviors will be automatically generated after the auto contract code generator is completed, without the developers needing to care about this part.
2.2 REST API for Contract#
We can use rest to get unsigned raw byte data for contract call, it can help to build an offline signature for contract.
can use this for example: rest
start the rest server:
go run ./examples/rest to get data:
curl -X POST \ 'http://localhost:8899/erc20/exec/transfer?isOffline=true&contract=5HKinTRKW9THEJxbQb22Nfyq9FPWNVZ9DQ2GEQ4Vg1LqTPuk' \ -H 'content-type: application/json' \ -d '{ "nonce":1, "chain_status":{ "spec_version":1, "tx_version":1, "block_hash":"0xc20f241b61039e5685d118c7fbc8b27210153c21eee7686a9466f22e01281114", "genesis_hash":"0xc20f241b61039e5685d118c7fbc8b27210153c21eee7686a9466f22e01281114" }, "contract":"5HKinTRKW9THEJxbQb22Nfyq9FPWNVZ9DQ2GEQ4Vg1LqTPuk", "origin":"5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY", "gas_limit":"500000000000", "args":{ "to":"5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty", "value":"100000000" }}'2.3 observer#
For a contract, we need observer events for the contract, can use observer to build a contract events observer service:
a complete example: observer
Also can take the dumper for example, it dump events by contacts to a db.
...
// create observer o := observer.New(logger, *flagURL) ctx, cancelFunc := context.WithCancel(context.Background())
... // other init functions...
// create a handler for process events in chain h := observer.NewEvtHandler() h = h.WithContractExecution(func(l log.Logger, height uint64, evt types.EventContractsContractExecution) { data := evt.Data
l.Debug("handler contract execution", "height", height)
// for golang we need process each diff types for event typ := metadata.GetEvtTypeIdx(data) switch typ { case 0: var transfer erc20.EventTransfer err := metaData.Spec.Events.DecodeEvt(metaData.NewCtxForDecode(data).WithLogger(l), &transfer) if err != nil { logger.Error("evt decode transfer error", "err", err, "height", height) } logger.Info("transfer event", "evt", transfer) case 1: var approve erc20.EventApproval err := metaData.Spec.Events.DecodeEvt(metaData.NewCtxForDecode(data).WithLogger(l), &approve) if err != nil { logger.Error("evt decode approve error", "err", err, "height", height) } logger.Info("approve event", "evt", approve) } })
// watcher events if err := o.WatchEvent(ctx, h); err != nil { logger.Error("watch event error", "err", err) return }
...2.4 Support for ERC20 Contract#
In github.com/patractlabs/go-patract/contracts/erc20, Complete ERC20 contract support.
...
erc20API := erc20.New(rpcAPI, contractAccountID) ctx := rpc.NewCtx(context.Background()).WithFrom(signature.TestKeyringPairAlice) // transfer alice to bob aliceTotal, err := erc20API.BalanceOf(ctx, test.AliceAccountID)
// transfer _, err = erc20API.Transfer(ctx, bob, types.NewBalanceByU64(100))
bobBalance, err := erc20API.BalanceOf(ctx, bob) aliceNewTotal, err := erc20API.BalanceOf(ctx, test.AliceAccountID)
...2.4 Tools#
We make some tools for developing constracts, like:
- https://github.com/patractlabs/go-patract/tree/master/tools/parser a parser for address
- https://github.com/patractlabs/go-patract/tree/master/tools/querier a querier for get some datas from chain
3. PatractPy Report#
For Unittest, should install Europa at first.
europa --versioneuropa 0.1.0-3f71403-x86_64-linux-gnuAll of test passed by Europa environment.
The design goal of PatractPy is to simplify the repetitive work of the interaction between developers and the contracts, and to monitor contract events, so as to provide a simplified way to quickly call and debug the contract. Therefore, the function of PatractPy will be similar to the api-contracts in polkadot.js. It can load contract instances according to the metadata of the contract, and automatically provide corresponding methods to the python contract instances according to the content of the metadata. Developers can call these generated methods to interact with the contract on the chain.
3.1 Basic Apis For Contracts#
As polkascan's Python Substrate Interface has provide some support to contract api, so we not need to improve the apis for contract calls, but there is some api to add:
SubstrateSubscriberis a subscriber support to subscribe data changes in chain, for example, the events in chain.get_contract_event_typeadd event decode support for contracts.
The basic api split into 2 parts:
- Contract, include:
- contractExecutor: This api could construct an extrinsic to call a contract, would be packed into a block and change state.
- contractReader: This api could construct a rpc request to call a contract, do not pack into a block and do not change any state.
- contractCreator: This api is used for
instantiatea contract and holding the Wasm code and metadata, receive following parameters:gas_limitendowmentdeployment_salt(saltparameter ininstantiate)
- ContractAPI: This api is used for
calla contract, is a wrapper forcontractExecutorandcontractReader, developers could use this api to react with contracts. This api could create a instance depends on the metadata, auto generate the contract access functions based on the contract. And the auto-gen functions receive the parameters which defined in contracts, besides receive following common parameters:gas_limitvalue(notice, if current call'spayableis false, thisvaluemust be0)
- ContractFactory: This api is used for constructing a contract instance in python, and developer could use this instance to access contract. This is a wrapper for
contractCreatorandContractAPI
- Observer, include:
- ContractObserver: This api is used for listen the events in contracts.
Based on the api provided above, Python developers can refer to the following case to access the contract. The following example shows a contract instance loaded through the metadata of ERC20. Developers can directly call methods like transfer that are automatically created by metadata to send transactions to the node or call the node's rpc to return the contract execution result. On the other hand, the following example also shows how to monitor the events of a contract:
import osfrom substrateinterface import SubstrateInterface, Keypairfrom patractinterface.contract import ContractAPI, ContractFactoryfrom patractinterface.observer import ContractObserver
def main(): # use [europa](https://github.com/patractlabs/europa) as test node endpoint, notice `type_registry` should set correctly. substrate=SubstrateInterface(url='ws://127.0.0.1:9944', type_registry_preset="default", type_registry={'types': {'LookupSource': 'MultiAddress'}}) # load deployer key alice = Keypair.create_from_uri('//Alice') bob = Keypair.create_from_uri('//Bob') # 1. load a contract from Wasm file and metadata.json file (Those files is complied by [ink!](https://github.com/paritytech/ink)) # in this example, we use `ink/example/erc20` contract as example. contract = ContractFactory.create_from_file( substrate=substrate, # should provide a subtrate endpoint code_file= os.path.join(os.path.dirname(__file__), 'res', 'erc20.wasm'), metadata_file= os.path.join(os.path.dirname(__file__), 'res', 'erc20.json') ) # upload code to chain directly res = contract.put_code(alice) print("update code hash{} res:{}".format(contract.code_hash.hex(), res.is_succes)) # 2. instantiate the uploaded code as a contract instance erc20_ins = contract.new(alice, 1000000 * (10 ** 15), endowment=2*10**10, gas_limit=20000000000, deployment_salt="0x12") # 2.1 create a observer to listen event observer = ContractObserver(erc20_ins.contract_address, erc20_ins.metadata, substrate) # 3. send a transfer call for this contract res = erc20_ins.transfer(alice, bob.ss58_address, 100000, gas_limit=20000000000) print('transfer res', res.is_succes)
def on_transfer(num, evt): print("on_transfer in {} : {} {} {}".format(num, evt['from'], evt['to'], evt['value']))
def on_approval(num, evt): print("on_approval in {} : {} {} {}".format(num, evt['owner'], evt['spender'], evt['value'])) # 4 set event callback observer.scanEvents(handlers={ 'Transfer': on_transfer, 'Approve': on_approval })
if __name__ == "__main__": main() pass3.2 ContractObserver#
ContractObserver can observer events for a contract:
substrate=SubstrateInterface(url="ws://127.0.0.1:9944", type_registry_preset='canvas')contract_metadata = ContractMetadata.create_from_file( metadata_file=os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'), substrate=substrate)observer = ContractObserver("0x8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48", contract_metadata, substrate)
# for some handlersobserver.scanEvents()The handler function can take the erc20 support as a example.
3.3 ERC20 API#
In addition to the normal use of metadata to construct contract objects to access the contract, developers can also package their own access methods to the contract according to the basic api provided by PatractPy.
ERC20 api provide a wrapper to erc20 contract exec, read and observer events, it can be a example for contracts api calling.
# init apisubstrate=SubstrateInterface(url="ws://127.0.0.1:9944", type_registry_preset='canvas')
contract_metadata = ContractMetadata.create_from_file( metadata_file=os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'), substrate=substrate)
alice = Keypair.create_from_uri('//Alice')bob = Keypair.create_from_uri('//Bob')
# erc20 apierc20 = ERC20.create_from_contracts( substrate= substrate, contract_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.wasm'), metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'))
# deplay a erc20 contracterc20.putAndDeploy(alice, 1000000 * (10 ** 15))
# read total supplytotal_supply = erc20.totalSupply()
# transfererc20.transferFrom(alice, fromAcc=alice.ss58_address, toAcc=bob.ss58_address, amt=10000)
erc20.transfer(alice, bob.ss58_address, 10000)
# get balancealice_balance = erc20.balanceOf(alice.ss58_address)
# approveerc20.approve(alice, spender=bob.ss58_address, amt=10000)
# get allowancealice_allowance = erc20.allowance(alice.ss58_address, bob.ss58_address)
ERC20Observer is a event observer for erc20 contract:
observer = ERC20Observer.create_from_address( substrate = substrate, contract_address = contract_address, metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'))
def on_transfer(num, fromAcc, toAcc, amt): logging.info("on_transfer in block[{}] : {} {} {}".format(num, fromAcc, toAcc, amt))
def on_approval(num, owner, spender, amt): logging.info("on_approval in block[{}] : {} {} {}".format(num, owner, spender, amt))
observer.scanEvents(on_transfer = on_transfer, on_approval = on_approval)3.4 Unittest Node Environment#
PatractPy can support write contract unittest by node environment.
At First We need install europa.
from patractinterface.contracts.erc20 import ERC20from patractinterface.unittest.env import SubstrateTestEnv
class UnittestEnvTest(unittest.TestCase): @classmethod def setUp(cls): # start env or use canvas for a 6s block cls.env = SubstrateTestEnv.create_europa(port=39944) cls.env.startNode()
cls.api = SubstrateInterface(url=cls.env.url(), type_registry_preset=cls.env.typ()) cls.alice = Keypair.create_from_uri('//Alice') cls.bob = Keypair.create_from_uri('//Bob')
cls.erc20 = ERC20.create_from_contracts( substrate= cls.substrate, contract_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.wasm'), metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json') ) cls.erc20.putAndDeploy(alice, 1000000 * (10 ** 15))
return
def tearDown(cls): cls.env.stopNode()
def test_transfer(self): self.erc20.transferFrom(alice, fromAcc=alice.ss58_address, toAcc=bob.ss58_address, amt=10000) # some more test case
if __name__ == '__main__': unittest.main()By example, we can use python to write testcase for some complex logics, by Europa, we can test the contracts for python scripts.
4. Recap of verification of v0.1 and v0.2#
v0.1 PatractGo
Initialize the project and complete the native contract interaction APIComplete the contract interaction API and HTTP contract interaction service based on metadata.jsonComplete contract status scan supportMonitor service support and complete the example to write into the databaseComplete ERC20 contract support and complete the corresponding command line toolsImprove sample code and unboxing documentation
v0.2 PatractPy
Initialize the project and complete the native contract interaction APIComplete the contract interaction API based on metadata.jsonComplete ERC20 contract supportImprove sample code and unboxing documentation
We will propose v0.3 and v0.4 later after some time for research.