Merge branch 'develop' of github.com:input-output-hk/cardano-sl into faucet

Author: rvl

CHANGELOG.md

@@ -1,18 +1,104 @@
 # CHANGELOG
 
-## Cardano SL 1.2.1 (Mainnet)
+## Cardano SL 1.3.0
+
+### Features
+
+- #### Cardano wallet API v1 for the exchanges
+  A subset of endpoints from the Cardano wallet API, used by cryptocurrency exchanges, was moved to the new 
+  Cardano REST API v1. The remaining endpoints will move from v0 to v1 in the next iteration of work. 
+  Cryptocurrency exchanges using the API are encouraged to update their integrations and move to API v1 
+  because the old API is now regarded as deprecated and will be removed in future versions. (CBR-101)
+
+- #### Subscription status added to node information
+  The endpoint that provides information about the node (`/api/v1/node-info`) was expanded to include
+  information about the node subscription status, which can be used to check if the Cardano node is 
+  connected to the network or not. (CBR-186)
+
+- #### Improved error message for missing charset in API calls
+  The generic message returned from Cardano node endpoint calls when the charset was not specified is replaced with 
+  a more descriptive error message to help users of the API troubleshoot the problem. (Wallet Backend - CBR-223)
+
+- #### Support for sending raw data on the network
+  It is now possible to send raw, CBOR serialized data to the network using the diffusion layer. Previously, the data 
+  had to be serialized before sending it to the network, which had a negative impact on performance. (CBR-277)
+
+### Specifications & documentation
+
+- #### Formal specification for new wallet backend
+  The formal specification of a wallet for Cardano (or any UTXO-based cryptocurrency) has been written and it is 
+  available [here](https://cardanodocs.com/technical/formal-specification-for-a-cardano-wallet/). (CBR-60)
+
+- Feedback about the current Wallet API has been collected from Exchanges (CBR-104).
+
+- Complete Peer Discovery (P2P) design (CDEC-157).
+
+- Specification of shared seed generation via VSS (CDEC-180).
+
+- Specification of Randomness Generation (CDEC-208).
+
+- As-is specifications of ATRedeem addresses (CDEC-366).
+
+### Fixes and improvements
+
+- #### Performance improvements for sending and receiving blocks
+  Sending and receiving blocks on the network now works better because of the following changes. First, 
+  deserialization performance has been improved by optimizing memory usage. Next, blocks are now downloaded 
+  concurrently without batching. Finally, block traversal is optimized by the introduction of 'forward links' which 
+  remove the need for header retrieval and serialization. (CDEC-49)
+
+- #### Optimised block storage
+  Block storage is now optimized by consolidating block and undo data in a single file. This change reduces 
+  disk use and improves performance when reading and writing blocks. In later Cardano versions, much greater 
+  optimizations for the storage of blocks will be introduced, so this is only an interim solution. (CDEC-293)
+
+- #### High (and recurrent) I/O traffic in wallet
+  I/O spikes in traffic were being caused by large logs being flushed. This issue has been fixed. (CBR-83)
+
+- #### Failure to reconnect to the network
+  Due to improper handling of DNS failures, Cardano node would sometimes fail to reconnect to the network after an 
+  internet connection was interrupted and would need to be restarted. This issue has been fixed. (CDEC-259)
+
+- #### Wrong time difference calculation between user’s computer and the network 
+  The endpoint (`/api/settings/time/difference`) for calculating the time difference between a user’s computer and 
+  Cardano network was returning an incorrect value in some cases. This was because the calculation was not properly 
+  handling the time needed to request current time from NTP servers and to get the response. As a result, some Daedalus 
+  users were prevented from using their wallet since Daedalus cannot be used if there is a time difference of more 
+  han 15 seconds. This issue has been fixed. (TSD-42)
+
+### Testing
+
+- Implement WalletActiveLayer & WalletPassiveLayer for wallet testing purposes (CBR-163).
+
+- Add integration deterministic tests for the Transaction endpoints (CBR-184).
+
+### Chores
+
+- Back port Timer to Pos.Diffusion.Subscription.Common (CDEC-243).
+
+- Message size limits should not be configurable (CDEC-260).
+
+- Upgrade to GHC 8.2.2 (CBR-51).
+
+- Fix AppVeyor hard limitation on Windows (CBR-268).
+
+- Fix tmux versions in demo-script (CO-295).
+
+- Clean script fails if file is missing (CO-316).
+
+## Cardano SL 1.2.1
 
 Bug fix release.
 
-- The wallet launcher now uses a lock file. This prevents problems on
+- The Wallet Launcher now uses a lock file. This prevents problems on
   Windows if upgrading Daedalus while the old version is still
   running. (DEVOPS-872)
 
 - Fix character encoding error in the connect script of the Docker
   image. (DEVOPS-877)
 
 
-## Cardano SL 1.2.0 (Mainnet)
+## Cardano SL 1.2.0 
 
 These are the most important code changes which were included in release 1.2.0.
 
@@ -38,14 +124,20 @@ These are the most important code changes which were included in release 1.2.0.
 
 - Add sorting and filtering capabilities to the wallet V1 API (CBR-20).
 
+### Specifications & documentation
+
+- Document the new Wallet V1 API (CBR-102, CBR-183, CO-105 & CBR-278).
+
+- Write a devops guide for the Exchanges (CBR-137).
+
 ### Bug fixes
 
 - Fix wallet creation and backup when there are non-latin characters in wallet name (R120-4).
 
 - Fix issue where Daedalus remains stuck at "Connecting to network..." screen after updating version (CBR-282, R120-17).
 
 
-## Cardano SL 1.1.1 (Mainnet)
+## Cardano SL 1.1.1
 
 Bug fix release.
 
@@ -56,7 +148,7 @@ Bug fix release.
    infrastructure, which fixes some issues in receiving bug reports.
 
 
-## Cardano SL 1.1.0 (Mainnet)
+## Cardano SL 1.1.0
 
 Most important code changes which made to release 1.1.0.
 

appveyor.yml

@@ -73,7 +73,7 @@ before_test:
 
 # Install rocksdb
 - git clone https://github.com/facebook/rocksdb.git --branch v4.13.5
-- ps: Start-FileDownload 'https://ci.appveyor.com/api/buildjobs/kbpteb8j55p6sa2m/artifacts/rocksdb%2Fbuild%2FRocksdb.zip' -FileName rocksdb.zip
+- ps: Start-FileDownload 'https://s3.eu-central-1.amazonaws.com/ci-static/serokell-rocksdb-haskell-325427fc709183c8fdf777ad5ea09f8d92bf8585.zip' -FileName rocksdb.zip
 - 7z x rocksdb.zip
 
 # CSL-1509: After moving the 'cardano-sl' project itself into a separate folder ('lib/'), the 'cardano-text.exe' executable fails on AppVeyor CI.

auxx/Main.hs

@@ -7,14 +7,14 @@ import           Universum
 import           Control.Exception.Safe (handle)
 import           Data.Maybe (fromMaybe)
 import           Formatting (sformat, shown, (%))
-import           Mockable (Production (..), runProduction)
 import qualified Network.Transport.TCP as TCP (TCPAddr (..))
 import qualified System.IO.Temp as Temp
 import           System.Wlog (LoggerName, logInfo)
 
 import qualified Pos.Client.CLI as CLI
 import           Pos.Context (NodeContext (..))
 import           Pos.Core (ConfigurationError, epochSlots)
+import           Pos.Core.Mockable (Production (..), runProduction)
 import           Pos.Crypto (ProtocolMagic)
 import           Pos.DB.DB (initNodeDBs)
 import           Pos.Infra.Diffusion.Types (Diffusion, hoistDiffusion)

auxx/cardano-sl-auxx.cabal

@@ -56,7 +56,6 @@ library
                      , cardano-sl-db
                      , cardano-sl-generator
                      , cardano-sl-infra
-                     , cardano-sl-networking
                      , cardano-sl-ssc
                      , cardano-sl-txp
                      , cardano-sl-update
@@ -133,7 +132,6 @@ executable cardano-auxx
                      , cardano-sl-core
                      , cardano-sl-crypto
                      , cardano-sl-infra
-                     , cardano-sl-networking
                      , cardano-sl-txp
                      , cardano-sl-util
                      , log-warper

auxx/src/AuxxOptions.hs

@@ -48,7 +48,7 @@ data AuxxStartMode
 
 data AuxxAction
     = Repl
-    | Cmd { cmd :: !Text }
+    | Cmd !Text
 
 ----------------------------------------------------------------------------
 -- Parse action

auxx/src/Command/Tx.hs

@@ -28,9 +28,6 @@ import qualified Data.Text as T
 import qualified Data.Text.IO as T
 import           Data.Time.Units (Microsecond, fromMicroseconds, toMicroseconds)
 import           Formatting (build, int, sformat, shown, stext, (%))
-import           Mockable (Mockable, SharedAtomic, SharedAtomicT, concurrently,
-                     currentTime, delay, forConcurrently, modifySharedAtomic,
-                     newSharedAtomic)
 import           System.Environment (lookupEnv)
 import           System.IO (BufferMode (LineBuffering), hClose, hSetBuffering)
 import           System.Wlog (logError, logInfo)
@@ -44,6 +41,9 @@ import           Pos.Core (BlockVersionData (bvdSlotDuration),
                      deriveFirstHDAddress, makePubKeyAddress, mkCoin)
 import           Pos.Core.Configuration (genesisBlockVersionData,
                      genesisSecretKeys)
+import           Pos.Core.Mockable (Mockable, SharedAtomic, SharedAtomicT,
+                     concurrently, currentTime, delay, forConcurrently,
+                     modifySharedAtomic, newSharedAtomic)
 import           Pos.Core.Txp (TxAux (..), TxIn (TxInUtxo), TxOut (..),
                      TxOutAux (..), txaF)
 import           Pos.Crypto (EncryptedSecretKey, ProtocolMagic, emptyPassphrase,

auxx/src/Mode.hs

@@ -26,7 +26,6 @@ import           Control.Lens (lens, makeLensesWith)
 import           Control.Monad.Reader (withReaderT)
 import           Control.Monad.Trans.Resource (transResourceT)
 import           Data.Conduit (transPipe)
-import           Mockable (Production)
 import           System.Wlog (HasLoggerName (..))
 
 import           Pos.Block.BListener (MonadBListener (..))
@@ -44,6 +43,10 @@ import           Pos.Core (Address, HasConfiguration, HasPrimaryKey (..),
                      IsBootstrapEraAddr (..), deriveFirstHDAddress,
                      largestPubKeyAddressBoot, largestPubKeyAddressSingleKey,
                      makePubKeyAddress, siEpoch)
+import           Pos.Core.JsonLog (CanJsonLog (..))
+import           Pos.Core.Mockable (Production)
+import           Pos.Core.Reporting (HasMisbehaviorMetrics (..),
+                     MonadReporting (..))
 import           Pos.Core.Slotting (HasSlottingVar (..), MonadSlotsData)
 import           Pos.Crypto (EncryptedSecretKey, PublicKey, emptyPassphrase)
 import           Pos.DB (DBSum (..), MonadGState (..), NodeDBs,
@@ -52,12 +55,9 @@ import           Pos.DB.Class (MonadDB (..), MonadDBRead (..))
 import           Pos.Generator.Block (BlockGenMode)
 import           Pos.GState (HasGStateContext (..), getGStateImplicit)
 import           Pos.Infra.Network.Types (HasNodeType (..), NodeType (..))
-import           Pos.Infra.Reporting (HasMisbehaviorMetrics (..),
-                     MonadReporting (..))
 import           Pos.Infra.Shutdown (HasShutdownContext (..))
 import           Pos.Infra.Slotting.Class (MonadSlots (..))
 import           Pos.Infra.Util.JsonLog.Events (HasJsonLogConfig (..))
-import           Pos.Infra.Util.TimeWarp (CanJsonLog (..))
 import           Pos.Launcher (HasConfigurations)
 import           Pos.Ssc.Types (HasSscContext (..))
 import           Pos.Txp (HasTxpConfiguration, MempoolExt, MonadTxpLocal (..),

auxx/src/Plugin.hs

@@ -18,10 +18,10 @@ import           Control.Monad.Except (ExceptT (..), withExceptT)
 import           Data.Constraint (Dict (..))
 import           Data.Time.Units (Second)
 import           Formatting (float, int, sformat, (%))
-import           Mockable (Delay, Mockable, delay)
 import           System.IO (hFlush, stdout)
 import           System.Wlog (CanLog, HasLoggerName, logInfo)
 
+import           Pos.Core.Mockable (Delay, Mockable, delay)
 import           Pos.Crypto (AHash (..), ProtocolMagic, fullPublicKeyF,
                      hashHexF)
 import           Pos.Infra.Diffusion.Types (Diffusion)

binary/cardano-sl-binary.cabal

@@ -35,12 +35,15 @@ library
                      , formatting
                      , hashable
                      , lens
+                     , micro-recursion-schemes
+                     , mtl
                      , safecopy
                      , safe-exceptions
                      , serokell-util
                      , tagged
                      , template-haskell
                      , text
+                     , text-format
                      , th-utilities
                      , time-units
                      , universum
@@ -86,6 +89,7 @@ test-suite test
   other-modules:
                        Spec
                        Test.Pos.Binary.BiSerialize
+                       Test.Pos.Binary.BiSizeBounds
                        Test.Pos.Binary.Cbor.CborSpec
                        Test.Pos.Binary.Helpers
                        Test.Pos.Binary.Helpers.GoldenRoundTrip
@@ -120,6 +124,7 @@ test-suite test
                      , quickcheck-instances
                      , safecopy
                      , serokell-util >= 0.1.3.4
+                     , tagged
                      , template-haskell
                      , text
                      , formatting