Skip to content

Optimizations to improve performance on complex conditional schemas #2466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
eef551d
Implementation of If Then Else
Saulzi Feb 27, 2020
51387a0
Fix playground example
nickgros Jul 12, 2021
fca3592
Fix resolveSchema to handle more cases
nickgros Jul 12, 2021
ba15fd3
Don't remove the root schema after validating
nickgros Jul 12, 2021
75665ca
Warn when encountering error in validation
nickgros Jul 12, 2021
23288ae
Add tests to cover error cases
nickgros Jul 12, 2021
4cf5f7f
Merge branch 'master' of https://github.com/rjsf-team/react-jsonschem…
nickgros Jul 12, 2021
78fc62c
Memoize withIdRefPrefix and prevent recompiling during form validation
nickgros Jul 15, 2021
58d3335
Fix tests
nickgros Jul 15, 2021
5e7a08d
Memoize all the things
nickgros Jul 16, 2021
667903b
Import only memoize from lodash
nickgros Jul 22, 2021
ab36d1e
Revert outdated docs reorganization
nickgros Jul 22, 2021
b00f49f
Revert `const` changes
nickgros Jul 22, 2021
833006a
Remove default value from enum example
nickgros Jul 22, 2021
8796c9d
Delete `allOf` rather than set to undefined. Fixes liveOmit issue.
nickgros Jul 22, 2021
b6c53c8
Merge branch 'master' of https://github.com/rjsf-team/react-jsonschem…
nickgros Jul 22, 2021
f72a62c
Added tests
Juansasa Aug 6, 2021
75c4d84
Changed resolve method's name
Juansasa Aug 9, 2021
a08c563
Added $ref tests
Juansasa Aug 13, 2021
07a43c8
Merge branch 'master' into conditional-support
nickgros Sep 1, 2021
52747fb
Merge branch 'master' of https://github.com/rjsf-team/react-jsonschem…
nickgros Feb 10, 2022
123c480
Merge branch 'conditional-support' of github.com:nickgros/react-jsons…
nickgros Feb 10, 2022
964f537
Merge branch 'if_then_else' of https://github.com/stakater/react-json…
nickgros Feb 10, 2022
544133b
Merge branch 'master' of https://github.com/rjsf-team/react-jsonschem…
nickgros Feb 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 79 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,85 @@ For more information on what themes we support, see [Using Themes](usage/themes)

disabled until https://github.com/rjsf-team/react-jsonschema-form/issues/1584 is resolved

## Useful samples
Sometimes you may want to trigger events or modify external state when a field has been focused, so you can pass an `onFocus` handler, which will receive the id of the input that is focused and the field value.

### Submit form programmatically
You can use the reference to get your `Form` component and call the `submit` method to submit the form programmatically without a submit button.
This method will dispatch the `submit` event of the form, and the function, that is passed to `onSubmit` props, will be called.

```js
const onSubmit = ({formData}) => console.log("Data submitted: ", formData);
let yourForm;

render((
<Form schema={schema}
onSubmit={onSubmit} ref={(form) => {yourForm = form;}}/>
), document.getElementById("app"));

yourForm.submit();
```

## Styling your forms

This library renders form fields and widgets leveraging the [Bootstrap](http://getbootstrap.com/) semantics. That means your forms will be beautiful by default if you're loading its stylesheet in your page.

You're not necessarily forced to use Bootstrap; while it uses its semantics, it also provides a bunch of other class names so you can bring new styles or override default ones quite easily in your own personalized stylesheet. That's just HTML after all :)

If you're okay with using styles from the Bootstrap ecosystem though, then the good news is that you have access to many themes for it, which are compatible with our generated forms!

Here are some examples from the [playground](https://rjsf-team.github.io/react-jsonschema-form/), using some of the [Bootswatch](http://bootswatch.com/) free themes:

![](https://i.imgur.com/1Z5oUK3.png)
![](https://i.imgur.com/IMFqMwK.png)
![](https://i.imgur.com/HOACwt5.png)

Last, if you really really want to override the semantics generated by the lib, you can always create and use your own custom [widget](advanced-customization.md#custom-widget-components), [field](advanced-customization.md#custom-field-components) and/or [schema field](advanced-customization.md#custom-schemafield) components.


## JSON Schema supporting status

This component follows [JSON Schema](http://json-schema.org/documentation.html) specs. We currently support JSON Schema-07 by default, but we also support other JSON schema versions through the [custom schema validation](https://react-jsonschema-form.readthedocs.io/en/latest/validation/#custom-schema-validation) feature. Due to the limitation of form widgets, there are some exceptions as follows:

* `additionalItems` keyword for arrays

This keyword works when `items` is an array. `additionalItems: true` is not supported because there's no widget to represent an item of any type. In this case it will be treated as no additional items allowed. `additionalItems` being a valid schema is supported.

* `anyOf`, `allOf`, and `oneOf`, or multiple `types` (i.e. `"type": ["string", "array"]`)

The `anyOf` and `oneOf` keywords are supported; however, properties declared inside the `anyOf/oneOf` should not overlap with properties "outside" of the `anyOf/oneOf`.

You can also use `oneOf` with [schema dependencies](dependencies.md#schema-dependencies) to dynamically add schema properties based on input data.

The `allOf` keyword is partially supported.

* `"additionalProperties":false` produces incorrect schemas when used with [schema dependencies](#schema-dependencies). This library does not remove extra properties, which causes validation to fail. It is recommended to avoid setting `"additionalProperties":false` when you use schema dependencies. See [#848](https://github.com/mozilla-services/react-jsonschema-form/issues/848) [#902](https://github.com/mozilla-services/react-jsonschema-form/issues/902) [#992](https://github.com/mozilla-services/react-jsonschema-form/issues/992)

## Handling of schema defaults

This library automatically fills default values defined the [JSON Schema](http://json-schema.org/documentation.html) as initial values in your form. This also works for complex structures in the schema. If a field has a default defined, it should always appear as default value in form. This also works when using [schema dependencies](#schema-dependencies).

Since there is a complex interaction between any supplied original form data and any injected defaults, this library tries to do the injection in a way which keeps the original intention of the original form data.

Check out the defaults example on the [live playground](https://mozilla-services.github.io/react-jsonschema-form/) to see this in action.

### Merging of defaults into the form data

There are three different cases which need to be considered for the merging. Objects, arrays and scalar values. This library always deeply merges any defaults with the existing form data for objects.

This are the rules which are used when injecting the defaults:

- When the is a scalar in the form data, nothing is changed.
- When the value is `undefined` in the form data, the default is created in the form data.
- When the value is an object in the form data, the defaults are deeply merged into the form data, using the rules defined here for the deep merge.
- Then the value is an array in the form data, defaults are only injected in existing array items. No new array items will be created, even if the schema has minItems or additional items defined.

### Merging of defaults within the schema

In the schema itself, defaults of parent elements are propagated into children. So when you have a schema which defines a deeply nested object as default, these defaults will be applied to children of the current node. This also merges objects defined at different levels together with the "deeper" not having precedence. If the parent node defines properties, which are not defined in the child, they will be merged so that the default for the child will be the merged defaults of parent and child.

For arrays this is not the case. Defining an array, when a parent also defines an array, will be overwritten. This is only true when arrays are used in the same level, for objects within these arrays, they will be deeply merged again.

## Tips and tricks
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what's happening here with these docs changes, they are unrelated to the PR and are also out commented. I guess this should be reverted?


- Custom field template: <https://jsfiddle.net/hdp1kgn6/1/>
- Multi-step wizard: <https://jsfiddle.net/sn4bnw9h/1/>
Expand Down
4 changes: 2 additions & 2 deletions packages/core/src/components/fields/SchemaField.js
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,8 @@ function SchemaFieldRender(props) {
const FieldTemplate =
uiSchema["ui:FieldTemplate"] || registry.FieldTemplate || DefaultTemplate;
let idSchema = props.idSchema;
const schema = retrieveSchema(props.schema, rootSchema, formData);
var schema = retrieveSchema(props.schema, rootSchema, formData);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming all const -> var related changes are due to the original PR being a bit old. In any case, const is highly preferable to var, so please revert all of these changes.


idSchema = mergeObjects(
toIdSchema(schema, null, rootSchema, formData, idPrefix),
idSchema
Expand Down Expand Up @@ -403,7 +404,6 @@ function SchemaFieldRender(props) {
</FieldTemplate>
);
}

class SchemaField extends React.Component {
shouldComponentUpdate(nextProps, nextState) {
return !deepEquals(this.props, nextProps);
Expand Down
8 changes: 7 additions & 1 deletion packages/core/src/components/widgets/BaseInput.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ function BaseInput(props) {
console.log("No id for", props);
throw new Error(`no id for props ${JSON.stringify(props)}`);
}
const {
var {
value,
readonly,
disabled,
Expand Down Expand Up @@ -66,6 +66,12 @@ function BaseInput(props) {
return props.onChange(value === "" ? options.emptyValue : value);
};

// If its a constant value then we want to disable the control and set the default value
if (typeof schema.const !== "undefined") {
value = schema.const;
disabled = true;
}

return [
<input
key={inputProps.id}
Expand Down
8 changes: 7 additions & 1 deletion packages/core/src/components/widgets/CheckboxWidget.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import PropTypes from "prop-types";
import { schemaRequiresTrueValue } from "../../utils";

function CheckboxWidget(props) {
const {
var {
schema,
id,
value,
Expand All @@ -22,6 +22,12 @@ function CheckboxWidget(props) {
// "const" or "enum" keywords
const required = schemaRequiresTrueValue(schema);

// If its a constant value then we want to disable the control and set the default value
if (typeof schema.const !== "undefined") {
value = schema.const;
disabled = true;
}

return (
<div className={`checkbox ${disabled || readonly ? "disabled" : ""}`}>
{schema.description && (
Expand Down
72 changes: 57 additions & 15 deletions packages/core/src/utils.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import React from "react";
import * as ReactIs from "react-is";
import mergeAllOf from "json-schema-merge-allof";
import fill from "core-js-pure/features/array/fill";
import union from "lodash/union";
import jsonpointer from "jsonpointer";
Expand Down Expand Up @@ -189,6 +188,8 @@ function computeDefaults(
} else if ("default" in schema) {
// Use schema defaults for this node.
defaults = schema.default;
} else if ("const" in schema) {
defaults = schema.const;
} else if ("$ref" in schema) {
// Use referenced schema defaults for this node.
const refSchema = findSchemaDefinition(schema.$ref, rootSchema);
Expand Down Expand Up @@ -666,15 +667,16 @@ export function resolveSchema(schema, rootSchema = {}, formData = {}) {
} else if (schema.hasOwnProperty("dependencies")) {
const resolvedSchema = resolveDependencies(schema, rootSchema, formData);
return retrieveSchema(resolvedSchema, rootSchema, formData);
} else if (schema.hasOwnProperty("allOf")) {
} else if (schema["allOf"]) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the reason for this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allOf may be set to undefined here: https://github.com/nickgros/react-jsonschema-form/blob/5e7a08dce80c0c1c85293ac3465c63d65693c6d4/packages/core/src/utils.js#L767

I think I changed that per a comment on the previous PR. Could use delete instead and revert this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching that line to delete actually fixed the liveOmit issue that I mentioned here. I'm not sure if it fixed the stack overflow error that you saw.

return {
...schema,
allOf: schema.allOf.map(allOfSubschema =>
retrieveSchema(allOfSubschema, rootSchema, formData)
),
};
} else {
// No $ref or dependencies attribute found, returning the original schema.
// No $ref, dependencies, or allOf attribute found, so there's nothing to resolve.
// Returning the original schema.
return schema;
}
}
Expand All @@ -697,28 +699,68 @@ export function retrieveSchema(schema, rootSchema = {}, formData = {}) {
return {};
}
let resolvedSchema = resolveSchema(schema, rootSchema, formData);
if ("allOf" in schema) {
try {
resolvedSchema = mergeAllOf({
...resolvedSchema,
allOf: resolvedSchema.allOf,
});
} catch (e) {
console.warn("could not merge subschemas in allOf:\n" + e);
const { allOf, ...resolvedSchemaWithoutAllOf } = resolvedSchema;
return resolvedSchemaWithoutAllOf;

while ("if" in resolvedSchema) {
// Note that if and else are key words in javascript so extract to variable names which are allowed
var {
if: expression,
then,
else: otherwise,
...resolvedSchemaLessConditional
} = resolvedSchema;

var conditionalSchema = isValid(expression, formData, rootSchema)
? then
: otherwise;

if (conditionalSchema) {
conditionalSchema = resolveSchema(
conditionalSchema,
rootSchema,
formData
);
}
resolvedSchema = mergeSchemas(
resolvedSchemaLessConditional,
conditionalSchema || {}
);
}

let allOf = resolvedSchema.allOf;

if (allOf) {
for (var i = 0; i < allOf.length; i++) {
let allOfSchema = allOf[i];

// if we see an if in our all of schema then evaluate the if schema and select the then / else, not sure if we should still merge without our if then else
if ("if" in allOfSchema) {
allOfSchema = isValid(allOfSchema.if, formData, rootSchema)
? allOfSchema.then
: allOfSchema.else;
}

if (allOfSchema) {
allOfSchema = resolveSchema(allOfSchema, rootSchema, formData); // resolve references etc.
resolvedSchema = {
...mergeSchemas(resolvedSchema, allOfSchema),
allOf: undefined,
};
}
}
}

const hasAdditionalProperties =
resolvedSchema.hasOwnProperty("additionalProperties") &&
resolvedSchema.additionalProperties !== false;

if (hasAdditionalProperties) {
return stubExistingAdditionalProperties(
resolvedSchema = stubExistingAdditionalProperties(
resolvedSchema,
rootSchema,
formData
);
}

return resolvedSchema;
}

Expand Down Expand Up @@ -998,7 +1040,7 @@ export function toIdSchema(
const idSchema = {
$id: id || idPrefix,
};
if ("$ref" in schema || "dependencies" in schema || "allOf" in schema) {
if ("$ref" in schema || "dependencies" in schema) {
const _schema = retrieveSchema(schema, rootSchema, formData);
return toIdSchema(_schema, id, rootSchema, formData, idPrefix);
}
Expand Down
25 changes: 18 additions & 7 deletions packages/core/src/validate.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ function createAjvInstance() {
multipleOfPrecision: 8,
schemaId: "auto",
unknownFormats: "ignore",
useDefaults: true,
});

// add custom formats
Expand Down Expand Up @@ -300,17 +301,27 @@ export function withIdRefPrefix(schemaNode) {
*/
export function isValid(schema, data, rootSchema) {
try {
// add the rootSchema ROOT_SCHEMA_PREFIX as id.
// add the rootSchema. if it has no $id, use ROOT_SCHEMA_PREFIX as the cache key.
// then rewrite the schema ref's to point to the rootSchema
// this accounts for the case where schema have references to models
// that lives in the rootSchema but not in the schema in question.
return ajv
.addSchema(rootSchema, ROOT_SCHEMA_PREFIX)
.validate(withIdRefPrefix(schema), data);
let rootSchemaAdded;
if (rootSchema.$id) {
rootSchemaAdded = !!ajv.getSchema(rootSchema.$id);
} else {
rootSchemaAdded = !!ajv.getSchema(ROOT_SCHEMA_PREFIX);
}

if (!rootSchemaAdded) {
if (rootSchema.$id) {
ajv.addSchema(rootSchema);
} else {
ajv.addSchema(rootSchema, ROOT_SCHEMA_PREFIX);
}
}
return ajv.validate(withIdRefPrefix(schema), data);
} catch (e) {
console.warn("Encountered error while validating schema", e);
return false;
} finally {
// make sure we remove the rootSchema from the global ajv instance
ajv.removeSchema(ROOT_SCHEMA_PREFIX);
Comment on lines -312 to -314
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the root schema being removed (before my change)?

Since we now need to run the validator frequently to evaluate conditional subschemas, keeping the root schema in the cache is essential for performance, especially for large schemas.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've figured it out: this function uses the same ajv instance as validateFormData, which has some advantages and some drawbacks. I think with this new implementation (not removing the root schema), validation breaks for certain schemas. I'm not happy with the performance of the current implementation, though, so I'll have to figure something else out.

Copy link
Contributor Author

@nickgros nickgros Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One cause of performance issues here is that we can't take advantage of AJV's cache because withIdRefPrefix creates a new object every time. We see a substantial improvement in performance if we memoize withIdRefPrefix, because validate sees the same schema and can use the ajv cache instead of recompiling. I'm not yet sure that memoization is 100% safe here, but it seems like a possible path forward.

Another thing I've noticed: we get different behavior with the current implementation if the root schema has an $id. When an $id is present, I think AJV totally ignores the root schema prefix when we call addSchema(rootSchema, ROOT_SCHEMA_PREFIX), so the removeSchema call never works. Not 100% on this either, though.

Copy link
Contributor Author

@nickgros nickgros Jul 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for $id issues, I've filed #2471, ideally would like to address it here (at the risk of increasing scope of this PR) since how RJSF uses AJV is closely related with adding if/then/else support

}
}
36 changes: 34 additions & 2 deletions packages/core/test/allOf_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,44 @@ describe("allOf", () => {
expect(node.querySelectorAll("input")).to.have.length.of(1);
});

it("should properly merge multiple schemas with refs", () => {
const schema = {
definitions: {
address: {
type: "object",
properties: {
street_address: { type: "string" },
city: { type: "string" },
state: { type: "string" },
},
required: ["street_address", "city", "state"],
},
},

allOf: [
{ $ref: "#/definitions/address" },
{
properties: {
type: { enum: ["residential", "business"] },
},
},
],
};

const { node } = createFormComponent({
schema,
});

expect(node.querySelectorAll("input")).to.have.length.of(3); // Schema 1
expect(node.querySelectorAll("select")).to.have.length.of(1); // Schema 2
});

it("should be able to handle incompatible types and not crash", () => {
const schema = {
type: "object",
properties: {
foo: {
allOf: [{ type: "string" }, { type: "boolean" }],
allOf: [{ type: "string" }, { type: "boolean" }], // this will basically pick up the last entry
},
},
};
Expand All @@ -44,6 +76,6 @@ describe("allOf", () => {
schema,
});

expect(node.querySelectorAll("input")).to.have.length.of(0);
expect(node.querySelectorAll("input")).to.have.length.of(1);
});
});
Loading