Skip to content

Disjoin Operation #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/core/config/Categories.json
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,7 @@
"ops": [
"Magic",
"Fork",
"Disjoin",
"Merge",
"Register",
"Label",
Expand Down
123 changes: 123 additions & 0 deletions src/core/operations/Disjoin.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
/**
* @author masq [[email protected]]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/

import Operation from "../Operation";
import Recipe from "../Recipe";
import Dish from "../Dish";

/**
* Disjoin operation
*/
class Disjoin extends Operation {

/**
* Disjoin constructor
*/
constructor() {
super();

this.name = "Disjoin";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not sure how I feel about the name Disjoin. Perhaps Concurrently?

this.flowControl = true;
this.module = "Default";
this.description = "Runs further operations in parallel on the same input until a Join operation is reached";
this.inputType = "string";
this.outputType = "string";
this.args = [
{
"name": "Split delimiter",
"type": "binaryShortString",
"value": "\\n"
},
Comment on lines +29 to +33
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this operation need to split? I think I'd prefer using Fork and then Disjoin. No need to have the same capability in both.

{
"name": "Merge delimiter",
"type": "binaryShortString",
"value": "\\n"
},
{
"name": "Ignore errors",
"type": "boolean",
"value": false
}
];
}

/**
* @param {Object} state - The current state of the recipe.
* @param {number} state.progress - The current position in the recipe.
* @param {Dish} state.dish - The Dish being operated on.
* @param {Operation[]} state.opList - The list of operations in the recipe.
* @returns {Object} The updated state of the recipe.
*/
async run(state) {
const opList = state.opList,
inputType = opList[state.progress].inputType,
outputType = opList[state.progress].outputType,
input = await state.dish.get(inputType),
ings = opList[state.progress].ingValues,
[splitDelim, mergeDelim, ignoreErrors] = ings,
subRecList = [];
let inputs = [],
i;

if (input)
inputs = input.split(splitDelim);

// Create subOpList for each tranche to operate on
// (all remaining operations unless we encounter a Merge)
for (i = state.progress + 1; i < opList.length; i++) {
if (opList[i].name === "Merge" && !opList[i].disabled) {
break;
} else {
const subRec = new Recipe();
subRec.addOperations(opList[i]);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addOperations takes a list of operations, should probably be
subRec.addOperations[opList[i]]

subRecList.push(subRec);
}
}

let output = "",
tmpProg = 0,
progress = 0;

state.forkOffset += state.progress + 1;

for (i = 0; i < subRecList.length; i++) {
const recipe = subRecList[i];

// Take a deep(ish) copy of the ingredient values
const ingValues = recipe.opList.map(op => JSON.parse(JSON.stringify(op.ingValues)));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use structuredClone instead


// Run recipe over each input
for (let j = 0; j < inputs.length; j++) {
// Baseline ing values for each tranche so that registers are reset
recipe.opList.forEach((op, i) => {
op.ingValues = JSON.parse(JSON.stringify(ingValues[i]));
});

const dish = new Dish();
dish.set(inputs[j], inputType);

try {
tmpProg = await recipe.execute(dish, 0, state);
} catch (err) {
if (!ignoreErrors) {
throw err;
}
tmpProg = err.progress + 1;
}
output += await dish.get(outputType) + mergeDelim;
}
progress += tmpProg;
}

state.dish.set(output, outputType);
state.progress += progress;
return state;
}


}

export default Disjoin;
1 change: 1 addition & 0 deletions test/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ import "./tests/operations/TranslateDateTimeFormat";
import "./tests/operations/Magic";
import "./tests/operations/ParseTLV";
import "./tests/operations/Media";
import "./tests/operations/Disjoin";

let allTestsPassing = true;
const testStatusCounts = {
Expand Down
81 changes: 81 additions & 0 deletions test/tests/operations/Disjoin.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/**
* Disjoin tests
*
* @author masq [[email protected]]
*
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
import TestRegister from "../../TestRegister";

TestRegister.addTests([
{
name: "Disjoin: nothing",
input: "",
expectedOutput: "",
recipeConfig: [
{
op: "Disjoin",
args: ["\n", "\n", false],
},
],
},
{
name: "Disjoin, Merge: nothing",
input: "",
expectedOutput: "",
recipeConfig: [
{
op: "Disjoin",
args: ["\n", "\n", false],
},
{
op: "Merge",
args: [],
},
],
},
{
name: "Disjoin, (expect) Error, Merge",
input: "1,2,3,4\n\n3,4,5,6",
expectedOutput: "Incorrect number of sets, perhaps you need to modify the sample delimiter or add more samples?",
recipeConfig: [
{
op: "Disjoin",
args: ["\n\n", "\n\n", false],
},
{
op: "Set Union",
args: ["\n\n", ","],
},
{
op: "Merge",
args: [],
},
],
},
{
name: "Disjoin: simple example",
input: "Test",
expectedOutput: "54 65 73 74\nVGVzdA==\n01010100 01100101 01110011 01110100\n",
recipeConfig: [
{"op": "Disjoin", "args": ["\\n", "\\n", "false"]},
{"op": "To Hex", "args": ["Space"]},
{"op": "To Base64", "args": ["A-Za-z0-9+/="]},
{"op": "To Binary", "args": ["Space"]},
],
},
//{
// name: "Disjoin, Conditional Jump, Encodings",
// input: "Some data with a 1 in it\nSome data with a 2 in it",
// expectedOutput: "U29tZSBkYXRhIHdpdGggYSAxIGluIGl0\n53 6f 6d 65 20 64 61 74 61 20 77 69 74 68 20 61 20 32 20 69 6e 20 69 74\n",
// recipeConfig: [
// {"op": "Fork", "args": ["\\n", "\\n", false]},
// {"op": "Conditional Jump", "args": ["1", false, "skipReturn", "10"]},
// {"op": "To Hex", "args": ["Space"]},
// {"op": "Return", "args": []},
// {"op": "Label", "args": ["skipReturn"]},
// {"op": "To Base64", "args": ["A-Za-z0-9+/="]}
// ]
//}
]);