splice: Script for complex splices #6980

ddustin · 2024-01-06T23:22:22Z

PR reviewers: It might be good to start here to learn the splice script syntax: https://github.com/ddustin/lightning/blob/ddustin/splice_script/doc/lightning-splice.7.md

New compiler for splice scripts that parses splice scripts, validates them, converts them to json and back again.

A test is included that puts some advanced and complex scripts into the parser and validates the results that come out.

Changelog-Added: Splice script parser — takes a custom splice query language to bundle multiple complex splices into a single task in a simple way.

vincenzopalazzo

Mh What do you mean by "Scripts"? and why now we need a "compiler" for splicing? 😕

I feel that I am missing somethings?

ddustin · 2024-01-08T19:14:29Z

Mh What do you mean by "Scripts"? and why now we need a "compiler" for splicing? 😕

I feel that I am missing somethings?

Proper documentation is coming but here's a quick summary:

The script isn't required to be used -- there will also be a json option and, of course, the lower level api is available.

When doing splices across multiple channels it quickly becomes unwieldy for users. Using the script is a nice shorthand to make it easier -- and also allows you to do "dry runs" and confirm it's doing what you'd like.

Say you wanted to take 50% of funds and splice them into three other channels, in splice script it's quite simple:

84ff50 -> 50%;
* -> cecf35;
* -> 4fe310;
* -> c14f60;

While working on it, it made sense to add a bunch of other useful things like being able to specify amounts as whole numbers, percentages, or * (split amount among all *s); channel queries; onchain bitcoin addresses; and the onchain wallet.

By building this more powerful part first it becomes trivial to implement the simpler splice commands. For instance:

Splice out 10 million sats

84ff50 -> 10M;

Splice in 10 million sats

wallet -> 10M;
* -> 84ff50;

Cross splice 10 million sats

84ff50 -> 10M;
* -> cecf35;

Pay 10M sats onchain using channel funds

84ff50 -> 10M+fee;
10M -> 1JfbZRwdDHKZmuiZgYArJZhcuuzuw2HuMu;

ddustin · 2024-11-01T20:18:38Z

OK, can we make this experimental, or a dev- command? I'm not convinced this is the way we want to go, but it's great for exercising all the nasty cases, and may well be useful for larger nodes.

Needs rebase because it was now "added v24.11" not "v24.08", sorry :(

I changed the command to "dev-splice," updated the added tag to 24.11, and fixed a handful of issues that came up from rebasing.

Can we push this one through once it passes CI @rustyrussell?

rustyrussell · 2024-11-10T11:58:49Z

Ack! Will apply once CI is happy!

rustyrussell · 2024-11-10T12:01:21Z

Accidentally told GH to update to latest master and it did a merge. Gah! Rebased instead.

Changelog-None

New compiler for splice scripts that parses splice scripts, validates them, converts them to json and back again. Changelog-Added: Splice script parser — takes a custom splice query language to bundle multiple complex splices into a single task in a simple way.

A test that puts some advanced and complex scripts into the parser and validates the results that come out.

A routine for getting a signature back out of an input’s list of pending signatures via pubkey search. This is needed for certain kinds of restarts as we lose our peer’s signature from memory but a copy is kept in the PSBT.

Changelog-None

Stores an array of channel_ids into the PSBT globals area. Changelog-None

`splice_signed` now searchs the PSBT for channel ids Changelog-Changed: `splice_signed` parameters are switched in order to make `channel_id` an optional parameter, enabling multi-splice-signatures.

This is needed to all multi-channel splices. When channeld can return the signatures to the user (based on signing order precedent), it now does from splice_update. Additionally, we move sending of the initial psbt from splice_init down to splice_update. This is also necessary for correct psbt diff detection during multi-channel splices. Changelog-Changed: splice_update can in some cases now return the remotely partiall signed psbt to the user, if so `signtures_secured` will be true.

On certain well timed restarts we lose their siganture from memory and don’t receive it from them. In these cases we can extract it from the PSBT directly.

The ability to stfu channels in bulk is required to do complex multi channel operations. When stfu’ing in this manner, the available funds at the moment of stfu is returned to the user. In order to cancel the stfu we also add a bulk tx_abort command. Changelog-Added: `stfu_channels` and `abort_channels` are added for bulk multi-channel splice commands. These allow the user to pause (and resume) multiple channels in place.

This is the sister command of addpsbtoutput. Adds inputs equal to or greater than the amount requests, reservers them, and reports important information back out to the user. Changelog-Added: New low-level RPC command addpsbtinput to fund PSBTs directly and help with complex splices & dual-opens.

When set this flag tells addpsbtoutput to add the intiator serial_id to the added output. Changelog-Changed: addpsbtoutput now allows serial_id to be set while adding which is needed for splicing and dual.

The command called “splice” can take a json payload or a ‘splice script’, process it into a list of ‘actions’ and then execute those actions. These actions include or will include everything you would want to do with a splice: * Splice into a channel * Splice out of a channel * Fund from wallet * Deposit to wallet * Send funds to bitcoin address Changelog-Added: A new magic “splice” command is added that can take a ‘splice script’ or json payload and perform any complex splice across multiple channels merging the result into a single transaction. Some features are disabled and will be added in time.

Adding schema and documentatino for how to use the scripting portion of the “splice” RPC command.

The fee maximum is used to issue a warning to the user their feerate is high in case they accidentally make a large donation to miners. During python testing the feerates are high on purpose so we raise the warning level to at least the penality feerate.

Tests that splice-in and splice-out work through the scripting process and confirm the resulting balances are correct.

If the user does not supply enough funds treat it as an abort instead of a channel warning. Also clean up some indentation while we’re there.

rustyrussell · 2024-11-11T02:54:20Z

Now a proper rebase, taking into account jsonrpc request API changes. Also minor fixes since it didn't bisect build before.

ddustin force-pushed the ddustin/splice_script branch 9 times, most recently from 5952c41 to 2640bd6 Compare January 7, 2024 03:32

ddustin mentioned this pull request Jan 7, 2024

splice: first pass on splice_out command #6777

Closed

ddustin force-pushed the ddustin/splice_script branch from 2640bd6 to 4e7edec Compare January 7, 2024 18:03

vincenzopalazzo reviewed Jan 8, 2024

View reviewed changes

ddustin added this to the v24.02 milestone Jan 13, 2024

ddustin force-pushed the ddustin/splice_script branch 5 times, most recently from 731048c to ab31aa7 Compare January 22, 2024 22:12

ddustin marked this pull request as ready for review January 22, 2024 22:13

ddustin force-pushed the ddustin/splice_script branch from ab31aa7 to e0b188a Compare January 22, 2024 22:14

ddustin requested review from rustyrussell and endothermicdev January 23, 2024 19:37

ddustin force-pushed the ddustin/splice_script branch 7 times, most recently from 8105050 to 737ffc2 Compare January 24, 2024 22:43

ddustin force-pushed the ddustin/splice_script branch from 444456e to f6b0b51 Compare November 1, 2024 20:17

rustyrussell force-pushed the ddustin/splice_script branch from 912211f to 42a1029 Compare November 10, 2024 12:00

ddustin added 21 commits November 11, 2024 10:57

mac: build warning about unused result

b5801fd

Changelog-None

splice: A splice-script test that tests complex corner cases

d9dfb00

A test that puts some advanced and complex scripts into the parser and validates the results that come out.

psbt: Add ability to extract signature

87dea60

A routine for getting a signature back out of an input’s list of pending signatures via pubkey search. This is needed for certain kinds of restarts as we lose our peer’s signature from memory but a copy is kept in the PSBT.

psbt: Add ability to set global PSBT values

4814c53

Changelog-None

psbt: Add ability to set channel_ids in psbt globals

4953abd

Stores an array of channel_ids into the PSBT globals area. Changelog-None

splice: Modify splice_signed to work with multiple channels

981d070

`splice_signed` now searchs the PSBT for channel ids Changelog-Changed: `splice_signed` parameters are switched in order to make `channel_id` an optional parameter, enabling multi-splice-signatures.

psbt: comment typo fix

bd2923b

splice: Extract remote sig from psbt if needed

344de67

On certain well timed restarts we lose their siganture from memory and don’t receive it from them. In these cases we can extract it from the PSBT directly.

addpsbtoutput: Add serial_id flag

4c9d099

When set this flag tells addpsbtoutput to add the intiator serial_id to the added output. Changelog-Changed: addpsbtoutput now allows serial_id to be set while adding which is needed for splicing and dual.

splice: Documentation for the “splice” command

f3c4331

Adding schema and documentatino for how to use the scripting portion of the “splice” RPC command.

peer_control: Don’t report removed inflights

9bef7b0

splice: Add test of “splice” script command

bf09184

Tests that splice-in and splice-out work through the scripting process and confirm the resulting balances are correct.

splice: Abort on insufficent funds

d7f84ab

If the user does not supply enough funds treat it as an abort instead of a channel warning. Also clean up some indentation while we’re there.

splice: Add more verbose reestablish debug logging

07a4655

splice: reestablish should only assume stfu in one case

fd25a60

rustyrussell force-pushed the ddustin/splice_script branch from 42a1029 to fd25a60 Compare November 11, 2024 02:53

rustyrussell merged commit dd956e2 into ElementsProject:master Nov 11, 2024
39 checks passed

daywalker90 mentioned this pull request Nov 17, 2024

msggen: add missing methods for 24.11 #7549

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

splice: Script for complex splices #6980

splice: Script for complex splices #6980

Uh oh!

ddustin commented Jan 6, 2024 •

edited

Loading

Uh oh!

vincenzopalazzo left a comment

Uh oh!

ddustin commented Jan 8, 2024

Uh oh!

ddustin commented Nov 1, 2024

Uh oh!

rustyrussell commented Nov 10, 2024

Uh oh!

rustyrussell commented Nov 10, 2024

Uh oh!

rustyrussell commented Nov 11, 2024

Uh oh!

Uh oh!

Uh oh!

splice: Script for complex splices #6980

splice: Script for complex splices #6980

Uh oh!

Conversation

ddustin commented Jan 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincenzopalazzo left a comment

Choose a reason for hiding this comment

Uh oh!

ddustin commented Jan 8, 2024

Uh oh!

ddustin commented Nov 1, 2024

Uh oh!

rustyrussell commented Nov 10, 2024

Uh oh!

rustyrussell commented Nov 10, 2024

Uh oh!

rustyrussell commented Nov 11, 2024

Uh oh!

Uh oh!

Uh oh!

ddustin commented Jan 6, 2024 •

edited

Loading