You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
4377: README: document package localisation r=deepfire a=deepfire
In the README:
1. Add a ToC.
2. Fix the structure, so the ToC actually makes sense.
3. Fix some minor RST syntax issues.
4. Some very minor fixes.
5. Expand the Development section with the workbench usage info.
6. Add `cabal-plan` and `graphmod` to the workbench shell.
7. Makes the node start on Darwin, when metric collection is enabled, by fixing #4380
Co-authored-by: Kosyrev Serge <[email protected]>
Co-authored-by: Michael Karg <[email protected]>
Copy file name to clipboardExpand all lines: README.rst
+176-49
Original file line number
Diff line number
Diff line change
@@ -15,8 +15,10 @@
15
15
</tr>
16
16
</table>
17
17
18
+
.. contents:: Contents
19
+
18
20
*************************
19
-
``cardano-node`` Overview
21
+
Overview of the ``cardano-node`` repository
20
22
*************************
21
23
22
24
Integration of the `ledger <https://github.com/input-output-hk/cardano-ledger-specs>`_, `consensus <https://github.com/input-output-hk/ouroboros-network/tree/master/ouroboros-consensus>`_,
@@ -39,8 +41,12 @@ Network Configuration, Genesis and Topology Files
39
41
40
42
The latest supported networks can be found at `<https://book.world.dev.cardano.org/environments.html>`_
41
43
42
-
How to build
43
-
============
44
+
****
45
+
Obtaining ``cardano-node``
46
+
****
47
+
48
+
Building from source
49
+
====
44
50
45
51
Documentation for building the node can be found `here <https://docs.cardano.org/getting-started/installing-the-cardano-node>`_.
46
52
@@ -54,15 +60,15 @@ You can download the latest version of ``cardano-node`` and ``cardano-cli``:
You can download `here <https://hydra.iohk.io/job/Cardano/cardano-node/linux.windows.cardano-node-win64/latest-finished>`_.
63
69
64
70
Instructions
65
-
------------
71
+
++++
66
72
67
73
The download includes cardano-node.exe and a .dll. To run the node with cardano-node run you need to reference a few files and directories as arguments. These can be copied from the cardano-node repo into the executables directory. The command to run the node on mainnet looks like this:
68
74
@@ -79,8 +85,12 @@ You can pull the docker image with the latest version of cardano-node from `here
79
85
80
86
docker pull inputoutput/cardano-node
81
87
82
-
``cardano-node``
83
-
================
88
+
****
89
+
Using ``cardano-node``
90
+
****
91
+
92
+
Command line summary: ``cardano-node``
93
+
====
84
94
This refers to the client that is used for running a node.
85
95
86
96
The general synopsis is as follows:
@@ -128,37 +138,26 @@ The general synopsis is as follows:
128
138
129
139
* ``--validate-db`` - Flag to revalidate all on-disk database files
130
140
131
-
Configuration ``.yaml`` files
132
-
=============================
141
+
Configuration
142
+
====
133
143
134
-
The ``--config`` flag points to a ``.yaml`` file that is responsible to configuring the logging & other important settings for the node. E.g. see the Byron mainnet configuration in this
144
+
The ``--config`` flag points to a ``.yaml`` (or a structurally equivalent ``.json``) file that is responsible to configuring the logging & other important settings for the node. E.g. see the Byron mainnet configuration in this
Some of the more important settings are as follows:
137
148
138
149
* ``Protocol: RealPBFT`` -- Protocol the node will execute
139
150
140
151
* ``RequiresNetworkMagic``: RequiresNoMagic -- Used to distinguish between mainnet (``RequiresNoMagic``) and testnets (``RequiresMagic``)
141
152
142
-
143
-
Logging
144
-
========
145
-
146
-
Logs are output to the ``logs/`` dir.
147
-
148
-
Profiling & statistics
149
-
======================
150
-
151
-
Profiling data and RTS run stats are stored in the ``profile/`` dir.
152
-
153
-
Please see ``scripts/README.md`` for how to obtain profiling information using the scripts.
154
-
155
153
Scripts
156
154
=======
157
155
158
156
Please see ``scripts/README.md`` for information on the various scripts.
159
157
160
-
``cardano-cli``
161
-
===============
158
+
****
159
+
Using ``cardano-cli``
160
+
****
162
161
163
162
A CLI utility to support a variety of key material operations (genesis, migration, pretty-printing..) for different system generations.
164
163
Usage documentation can be found at ``cardano-cli/README.md``.
@@ -172,6 +171,9 @@ The general synopsis is as follows:
172
171
> NOTE: the exact invocation command depends on the environment. If you have only built ``cardano-cli``, without installing it, then you have to prepend ``cabal run -- ``
173
172
before ``cardano-cli``. We henceforth assume that the necessary environment-specific adjustment has been made, so we only mention ``cardano-cli``.
174
173
174
+
Command line options: ``cardano-cli``
175
+
====
176
+
175
177
The subcommands are subdivided in groups, and their full list can be seen in the output of ``cardano-cli --help``.
176
178
177
179
All subcommands have help available. For example:
@@ -194,11 +196,8 @@ All subcommands have help available. For example:
194
196
-h,--help Show this help text
195
197
196
198
197
-
Genesis operations
198
-
==================
199
-
200
-
Generation
201
-
----------
199
+
Genesis generation
200
+
====
202
201
203
202
The Byron genesis generation operations will create a directory that contains:
204
203
@@ -322,7 +321,7 @@ Local node queries
322
321
323
322
You can query the tip of your local node via the ``get-tip`` command as follows
324
323
325
-
1. Open `tmux`
324
+
1. Open ``tmux``
326
325
2. Run ``cabal build cardano-node``
327
326
3. Run ``./scripts/lite/shelley-testnet.sh example``
328
327
4. Run ``export CARDANO_NODE_SOCKET_PATH=/cardano-node/example/socket/node-1-socket
@@ -365,7 +364,7 @@ The mandatory arguments are ``--mainnet | --testnet-magic``, ``signing-key``, ``
365
364
366
365
The remaining arguments are optional parameters you want to update in your update proposal.
367
366
368
-
You can also check your proposal's validity using the `validate-cbor` command. See: `Validate CBOR files`_.
367
+
You can also check your proposal's validity using the ``validate-cbor`` command. See: `Validate CBOR files`_.
369
368
370
369
See the `Byron specification <https://hydra.iohk.io/job/Cardano/cardano-ledger-specs/byronLedgerSpec/latest/download-by-type/doc-pdf/ledger-spec>`_
371
370
for more details on update proposals.
@@ -413,54 +412,180 @@ Byron vote submission:
413
412
(--mainnet | --testnet-magic NATURAL)
414
413
--filepath UpdateProposalVoteFile
415
414
415
+
****
416
416
Development
417
-
===========
417
+
****
418
+
419
+
Overview
420
+
====
421
+
422
+
The ``cardano-node`` development is primarily based on the Nix infrastructure (https://nixos.org/ ), which enables packaging, CI, development environments and deployments.
423
+
424
+
On how to set up Nix for ``cardano-node`` development, please see `Building Cardano Node with nix <https://github.com/input-output-hk/cardano-node/tree/master/doc/getting-started/building-the-node-using-nix.md>`_.
425
+
426
+
Workbench: a local cluster playground
427
+
====
428
+
429
+
You can quickly spin up a local cluster (on Linux and Darwin), based on any of a wide variety of configurations, and put it under a transaction generation workload -- using the ``workbench`` environment:
430
+
431
+
1. Optional: choose a workbench profile:
432
+
- ``default`` stands for a light-state, 6-node cluster, under saturation workload, indefinite runtime
433
+
- ``ci-test`` is the profile run in the node CI -- very light, just two nodes and short runtime
434
+
- ``devops`` is an unloaded profile (no transaction generation) with short slots -- ``0.2`` sec.
435
+
- ..and many more -- which can be either:
436
+
- listed, by ``make ps``
437
+
- observed at their point of definition: `nix/workbench/profiles/prof1-variants.jq <https://github.com/input-output-hk/cardano-node/tree/master/nix/workbench/profiles/prof1-variants.jq#L333-L526>`_
438
+
2. Optional: select mode of operation, by optionally providing a suffix:
439
+
- default -- no suffix -- just enter the workbench shell, allowing you to run ``start-cluster`` at any time. Binaries will be built locally, by ``cabal``.
440
+
- ``autostay`` suffix -- enter the workbench shell, start the cluster, and stay in the shell afterwards. Binaries will be built locally, by ``cabal``.
441
+
- ``autonix`` suffix -- enter the workbench shell, start the cluster. All binaries will be provided by the Nix CI.
442
+
- ..there are other modes, as per `lib.mk <https://github.com/input-output-hk/cardano-node/tree/master/lib.mk>`_
443
+
3. Enter the workbench shell for the chosen profile & mode:
444
+
``make <PROFILE-NAME>`` or ``make <PROFILE-NAME>-<SUFFIX>`` (when there is a suffix).
445
+
4. Optional: start cluster:
446
+
Depending on the chosen mode, your cluster might already start, or you are expected to start it yourself, using ``start-cluster``.
447
+
448
+
The workbench services are available only inside the workbench shell.
449
+
450
+
Using Cabal
451
+
----
452
+
453
+
By default, all binaries originating in the ``cardano-node`` repository are available to ``cabal build`` and ``cabal run``, unless the workbench was entered using one of the pure ``*nix`` modes. Note that in all cases, the dependencies for the workbench are supplied though Nix and have been built/tested on CI.
454
+
455
+
**Dependency localisation** -or- *Cabal&Nix for painless cross-repository work*
456
+
----
457
+
458
+
The Cabal workflow described above only extends to the repository-local packages. Therefore, ordinarily, to work on ``cardano-node`` dependencies in the context of the node itself, one needs to go through an expensive multi-step process -- with committing, pushing and re-pinning of the dependency changes.
459
+
460
+
The **dependency localisation** workflow allows us to pick a subset of leaf dependencies of the ``cardano-node`` repository, and declare them *local* -- so they can be iterated upon using the ``cabal build`` / ``cabal run`` of ``cardano-node`` itself. This cuts development iteration time dramatically and enables effective cross-repo development of the full stack of Cardano packages.
461
+
462
+
Without further ado (**NOTE**: *the order of steps is important!*):
463
+
464
+
1. Ensure that your ``cardano-node`` checkout is clean, with no local modifications. Also, ensure that you start outside the node's Nix shell.
465
+
2. Check out the repository with the dependencies, *beside* the ``cardano-node`` checkout. You have to check out the git revision of the dependency used by your ``cardano-node`` checkout -- as listed in ``cardano-node/cabal.project``.
466
+
- we'll assume the ``ouroboros-network`` repository
467
+
- ..so a certain parent directory will include checkouts of both ``ouroboros-network`` and ``cardano-node``, at the same level
468
+
- ..and the git revision checked out in ``ouroboros-network`` will match the corresponding ``source-repository-package`` clause in ``cardano-node/cabal.project``.
469
+
- Extra point #1: you can localise/check out several dependency repositories
470
+
- Extra point #2: for the dependencies that are not listed in ``cabal.project`` of the node -- how do you determine the version to check out? You can ask the workbench shell:
471
+
1. Temporarily enter the workbench shell
472
+
2. Look for the package version in ``ghc-pkg list``
473
+
3. Use that version to determine the git revision of the dependency's repository (using a tag or some special knowledge about the version management of said dependency).
474
+
3. Enter the workbench shell, as per instructions in previous sections -- or just a plain Nix shell.
475
+
4. Ensure you can build ``cardano-node`` with Cabal: ``cabal build exe:cardano-node``. If you can't something else is wrong.
476
+
5. Determine the *leaf dependency set* you will have to work on. The *leaf dependency set* is defined to include the target package you want to modify, and its reverse dependencies -- that is, packages that depend on it (inside the dependency repository).
477
+
- let's assume, for example, that you want to modify ``ouroboros-consensus-shelley``
478
+
- ``ouroboros-consensus-shelley`` is not a leaf dependency in itself, since ``ouroboros-consensus-cardano`` (of the same ``ouroboros-network`` repository) depends on it -- so the *leaf dependency set* will include both of them.
479
+
- you might find out that you have to include a significant fraction of packages in ``ouroboros-network`` into this *leaf dependency set* -- do not despair.
480
+
- if the *leaf dependency set* is hard to determine, you can use ``cabal-plan`` -- which is included in the workbench shell (which you, therefore, have to enter temporarily):
This command will produce a HUGE ``deps.png`` file, which will contain the entire chart of the project dependencies. The important part to look for will be the subset of packages highlighted in red -- those, which belong to the ``ouroboros-network`` repository. This will be the full *leaf dependency set*.
486
+
6. Edit the ``cardano-node/cabal.project`` as follows:
487
+
- for the *leaf dependency set*
488
+
1. in the very beginning of the ``cabal.project``, add their relative paths to the ``packages:`` section, e.g.:
489
+
.. code-block:: console
490
+
491
+
packages:
492
+
cardano-api
493
+
cardano-cli
494
+
...
495
+
trace-resources
496
+
trace-forward
497
+
../ouroboros-network/ouroboros-consensus-shelley
498
+
../ouroboros-network/ouroboros-consensus-cardano
499
+
500
+
2. in the corresponding ``source-repository-package`` section, comment out mentions of the packages, e.g.:
7. The two packages have now became **local** -- when you try ``cabal build exe:cardano-node`` now, you'll see that Cabal starts to build these dependencies you just localised. Hacking time!
521
+
522
+
Hoogle
523
+
----
524
+
525
+
The workbench shell provides ``hoogle``, with a local database for the full set of dependencies:
Ouroboros.Consensus.HardFork.Combinator data family TxId tx :: Type
540
+
-- plus more results not shown, pass --count=20 to see more
541
+
542
+
Supplementary tooling
543
+
====
418
544
419
545
GHCID
420
-
-----
546
+
----
421
547
422
548
run *ghcid* with: ``ghcid -c "cabal repl exe:cardano-node --reorder-goals"``
423
549
424
550
Haskell Language Server
425
-
-----------------------
551
+
----
426
552
427
553
When using Haskell Language Server with Visual Studio Code, you may find that
428
-
`HLINT annotations are ignored<https://github.com/haskell/haskell-language-server/issues/638>`.
554
+
`HLINT annotations are ignored<https://github.com/haskell/haskell-language-server/issues/638>`_.
429
555
430
-
To work around this, you may run the script `./scripts/reconfigure-hlint.sh` to generate a `.hlint.yaml`
556
+
To work around this, you may run the script ``./scripts/reconfigure-hlint.sh`` to generate a ``.hlint.yaml``
431
557
file with HLINT ignore rules derived from the source code.
432
558
433
559
Testing
434
-
========
560
+
====
435
561
436
562
``cardano-node`` is essentially a container which implements several components such networking, consensus, and storage. These components have individual test coverage. The node goes through integration and release testing by Devops/QA while automated CLI tests are ongoing alongside development.
437
563
438
564
Developers on ``cardano-node`` can `launch their own testnets <doc/getting-started/launching-a-testnet.md>`_ or `run the chairman tests <doc/getting-started/running-chairman-tests.md>`_ locally.
439
565
440
-
Chairman tests
441
-
--------------
442
566
443
567
Debugging
444
-
=========
568
+
====
445
569
446
570
Pretty printing CBOR encoded files
447
-
----------------------------------
571
+
----
448
572
449
573
It may be useful to print the on chain representations of blocks, delegation certificates, txs and update proposals. There are two commands that do this (for any cbor encoded file):
Native tokens is a new feature that enables the transacting of multi-assets on Cardano. Native tokens are now supported on mainnet and users can transact with ada, and an unlimited number of user-defined (custom) tokens natively.
466
591
@@ -472,10 +597,12 @@ To help you get started we have compiled a handy list of resources:
472
597
473
598
You can also read more about `native tokens and how they compare to ada and ERC20 <https://github.com/input-output-hk/cardano-ledger-specs/blob/master/doc/explanations/features.rst>`_. Browse native tokens created on the Cardano blockchain and see their transactions in an interactive dashboard that allows filtering and searching: nativetokens.da.iogservices.io.
474
599
600
+
****
475
601
API Documentation
476
-
=================
602
+
****
603
+
477
604
The API documentation is published `here <https://input-output-hk.github.io/cardano-node/>`_.
478
605
479
-
The documentation is built with each push, but is only published from `master` branch. In order to
480
-
test if the documentation is working, build the documentation locally with `./scripts/haddocs.sh` and
481
-
open `haddocks/index.html` in the browser.
606
+
The documentation is built with each push, but is only published from ``master`` branch. In order to
607
+
test if the documentation is working, build the documentation locally with ``./scripts/haddocs.sh`` and
![iohk-bors[bot]](https://avatars.githubusercontent.com/in/17473?v=4&size=40){kind=avatar}
0 commit comments