JSON Schema

Dr. Greg Bernstein

November 12th, 2021

Input Validation

In the Browser App

  • Simple input checking of input fields, to more complicated checking of multiple input fields.

  • Web apps that load data may need more elaborate type check of more complicated data structures in JSON format.

On the Server

  • JSON data from a client can never be trusted

  • This data must always be checked for general and application specific requirements and constraints

  • Performance and security can be very important, in addition we need to be very clear about what our APIs will accept

Quick Check at NPM.js I

[https://www.npmjs.com/package/@hapi/joi](https://www.npmjs.com/package/@hapi/joi)

hapi

Quick Check at NPM.js II

https://www.npmjs.com/package/ajv

ajv

JSON Schema

A way to specify the contents of a JSON file or message or JavaScript object/Array.

Examples Archive: JSONSchema.zip

Readings/References

Checking JS Objects

  • How can we tell if a JavaScript Object is valid in our application context?

  • Write a test function program to check for the presence and/or absence of various fields, and constraints on those fields

  • Seems like a lot of folks would need this functionality. Maybe someone else has looked at this before…

Specifying the Contents of JSON

  • In order to check the validity of a JS Object we need a way to specify what should and should not be in that object.

  • We need a language to describe JS Objects. Such a language used to describe things is called a schema. They can take a number of forms.

  • The schema we will use to describe JS Objects/JSON will be written in JSON! The particular flavor we will use is very popular and is known as JSON Schema

Tools for JSON Schema Development

  • Online JSON Schema Validator copy and paste your schema and sample data and they will let you know if it validates.

  • Command line utility based on AJV

    1. npm install -g ajv-cli
    2. Testing a JSON schema against a JSON data file:

    ajv -s yourSchema.json -d yourData.json

Schema “Declaration”

Need to let the world know that this is a JSON scheme and good to identify it

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://grotto-networking.com/example1.schema.json",
  "title": "My First Schema",
  "description": "A Schema that requires a JSON object",
  "type": "object"
}

where title and description are optional. type is extremely important!

JSON Schema type 1

From Types

  • string – Can impose limits (min/max) on length, check for standard formats such as email, IP addresses, use regular expressions to check, enum (restrict string to a set of values), etc…

  • Numeric types – Can limit to integer or leave as general number, can impose range restrictions and the like.

  • object – Must be an object, can specify its properties, which are required, and whether additionalProperties are allowed

JSON Schema type 2

From Types

  • array – Must be an array, can specify what its items must be like as either lists or tuples.

  • boolean – Must be a boolean

Example 1

Use the previous schema to test the following JSON

ex1Test1.json

{"name": "Dr. B.", "course": "CS anything"}

ex1Test2.json

[1, 2, 3, 4]

Example 1 Result

AJV command line

Validation Rules

Validation Keywords for Any Instance Type

From schema validation

  1. type – MUST be one of the six primitive types (“null”, “boolean”, “object”, “array”, “number”, or “string”), or “integer”

  2. enum – MUST be an array. This array SHOULD have at least one element. Elements in the array SHOULD be unique. An instance validates successfully against if its value is equal to one of the elements in the array

  3. const – is functionally equivalent to an “enum” with a single value.

Validation Keywords for Numeric Instances (number and integer)

From schema validation

  1. multipleOf
  2. maximum
  3. exclusiveMaximum
  4. minimum
  5. exclusiveMinimum

Validation Keywords for Strings

From schema validation

  1. maxLength
  2. minLength
  3. pattern

Validation Keywords for Arrays

From schema validation

  1. maxItems
  2. minItems
  3. uniqueItems
  4. maxContains
  5. minContains

Validation Keywords for Objects

From schema validation

  1. maxProperties
  2. minProperties
  3. required
  4. dependentRequired
  5. Also see additionalProperties

AJV Use

References

What Does it Do?

Ajv takes a schema for your JSON data and converts it into a very efficient JavaScript code that validates your data according to the schema. To create schema you can use either JSON Schema or JSON Type Definition…

AJV Example w/o “format”

From AJV Guide adapted to ES6 modules

import Ajv from 'ajv';
const ajv = new Ajv() // options can be passed, e.g. {allErrors: true}

const schema = {
  type: "object",
  properties: {
    foo: {type: "integer"},
    bar: {type: "string"}
  },
  required: ["foo"],
  additionalProperties: false
}

const validate = ajv.compile(schema)

const data = {
  foo: 1,
  bar: "abc"
}

const valid = validate(data)
if (!valid) console.log(validate.errors)

Formats

From JSON Schema Validation

Schema Formats

Using format with AJV 1

From Format Validation

From version 7 Ajv does not include formats defined by JSON Schema specification - these and several other formats are provided by ajv-formats plugin.

Example Club Validation

import { readFile } from "fs/promises"; // To read external schema files
import Ajv from 'ajv';
import addFormats from "ajv-formats" // For Schema "format"

// Read in Schemas
const activitySchema = JSON.parse(
  await readFile(new URL("./activitySchema.json", import.meta.url))
);
const applicantSchema = JSON.parse(
  await readFile(new URL("./applicantSchema.json", import.meta.url))
);
let ajv = new Ajv();
addFormats(ajv);
let activityValidate = ajv.compile(activitySchema);
let applicantValidate = ajv.compile(applicantSchema);
// use activityValidate and applicantValidate wherever needed.

More Examples

Example 2: Object with Properties

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": "http://grotto-networking.com/example2.schema.json",
    "title": "Windsurf Foil",
    "description": "For the most fun use a foil",
    "type": "object",
    "properties": {
        "frontWing": {
            "description": "Length of front wing in cm",
            "type": "integer"
        },
        "rearWing": {
            "description": "Length of rear wing in cm",
            "type": "integer"
        },
        "color": {
            "description": "Color of the foil",
            "type": "string"
        },
        "material": {
            "type": "string"
        }
    },
    "required": ["frontWing", "rearWing"],
    "additionalProperties": false
}

Example 2 Data Part 1

ex2Test1.json:

 {   "frontWing": 80,
     "rearWing": 40,
     "color": "black",
     "material": "carbon fiber" }

ex2Test2.json

 {   "frontWing": 80,
     "color": "black",
     "material": "carbon fiber" }

Example 2 Data Part 2

ex2Test3.json:

 {   "frontWing": 80,
     "color": "black",
     "material": "carbon fiber",
     "random": 42 }

ex2Test4.json

 {   "frontWing": "80 or so",
     "rearWing": 40,
     "color": "black",
     "material": "carbon fiber" }

Example 3: Array of Objects

Network Demands demandSchema.json

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "id": "http://grotto-networking.com/schemas/basicDemand",
    "title": "Demand collection format base",
    "descriptions": "Demand format for import and export. By Dr. Greg M. Bernstein",
    "type": "object",
    "properties": {
        "id": {
            "type": "string"
        },
        "desc": {
            "type": "string"
        },
        "demandList": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "source": {
                        "type": "string"
                    },
                    "target": {
                        "type": "string"
                    },
                    "demand": {
                        "type": "number",
                        "minimum": 0
                    }
                },
                "required": ["source", "target", "demand"]
            }

        }
    },
    "required": ["demandList"]
}

Example 3, Example Data

ChineseDemands4.json

{
"desc": "A few demands for my optical network for testing",

"demandList":[
    {"source": "n3", "target": "n51", "demand": 2.0},
    {"source": "n8", "target": "n21", "demand": 1.0},
    {"source": "n6", "target": "n23", "demand": 1.0},
    {"source": "n7", "target": "n36", "demand": 1.0}
    ]
}

Example 4 A Data Network Schema

netSchema.json

{
    "$schema": "http://json-schema.org/schema#",
    "title": "Network Visualization Format",
    "descriptions": "Network Visualization format compatible with NetworkX JSON graph",
    "definitions": {
        "node": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "string",
                    "minLength": 1
                },
                "type": {"type": "string"},
                "x": {"type": "number"},
                "y": {"type": "number"},
                "layer": {"type": "string"},
                "ipv4": {"type": "string"},
                "mac": {"type": "string"}
            },
            "required": ["id", "x", "y"]
        },
        "link": {
            "type": "object",
            "properties": {
                "source": {
                    "type": "string"
                },
                "target": {
                    "type": "string"
                },
                "capacity": {
                    "type": "number",
                    "minimum": 0
                },
                "weight": {
                    "type": "number",
                    "minimum": 0
                },
                "ports": {
                    "type": "object"
                }
            },
            "required": ["source", "target"]
        }
    },

    "type": "object",
    "properties": {
        "nodes": {
            "type": "array",
            "items": {
                "$ref": "#/definitions/node"
            }
        },
        "links": {
            "type": "array",
            "items": {
                "$ref": "#/definitions/link"
            }
        },
        "graph": {"type": "object",
            "properties": {
                "name": {"type": "string"},
                "desc": {"type": "string"}
            }},
        "directed": {"type": "boolean"},
        "multigraph": {"type": "boolean"}
    },
    "required": ["nodes"]
}

Example 4 Sample Data

ExNetwithLoops1B.json

{
    "directed": false,
    "multigraph": false,
    "graph": {
        "name": "ExNetwithLoops1B",
        "desc": "A Mininet network generation example network"
    },
    "nodes": [
        {
            "ipv4": "10.0.0.2",
            "mac": "00:00:00:00:00:02",
            "y": 358,
            "x": 456,
            "type": "host",
            "id": "H8"
        },
        {
            "ipv4": "10.0.0.3",
            "mac": "00:00:00:00:00:03",
            "y": 211,
            "x": 607,
            "type": "host",
            "id": "H9"
        },
        {
            "ipv4": "10.0.0.4",
            "mac": "00:00:00:00:00:04",
            "y": 123,
            "x": 125,
            "type": "host",
            "id": "H2"
        },
        {
            "ipv4": "10.0.0.5",
            "mac": "00:00:00:00:00:05",
            "y": 176,
            "x": 171,
            "type": "host",
            "id": "H3"
        },
        {
            "y": 142,
            "x": 209,
            "type": "switch",
            "id": "S1"
        },
        {
            "ipv4": "10.0.0.6",
            "mac": "00:00:00:00:00:06",
            "y": 61,
            "x": 204,
            "type": "host",
            "id": "H1"
        },
        {
            "ipv4": "10.0.0.7",
            "mac": "00:00:00:00:00:07",
            "y": 267,
            "x": 399,
            "type": "host",
            "id": "H6"
        },
        {
            "ipv4": "10.0.0.8",
            "mac": "00:00:00:00:00:08",
            "y": 378,
            "x": 373,
            "type": "host",
            "id": "H7"
        },
        {
            "y": 157,
            "x": 593,
            "type": "switch",
            "id": "S5"
        },
        {
            "y": 323,
            "x": 419,
            "type": "switch",
            "id": "S4"
        },
        {
            "y": 107,
            "x": 408,
            "type": "switch",
            "id": "S3"
        },
        {
            "ipv4": "10.0.0.9",
            "mac": "00:00:00:00:00:09",
            "y": 273,
            "x": 155,
            "type": "host",
            "id": "H4"
        },
        {
            "y": 247,
            "x": 230,
            "type": "switch",
            "id": "S2"
        },
        {
            "ipv4": "10.0.0.10",
            "mac": "00:00:00:00:00:10",
            "y": 67,
            "x": 364,
            "type": "host",
            "id": "H11"
        },
        {
            "ipv4": "10.0.0.11",
            "mac": "00:00:00:00:00:11",
            "y": 325,
            "x": 209,
            "type": "host",
            "id": "H5"
        },
        {
            "ipv4": "10.0.0.12",
            "mac": "00:00:00:00:00:12",
            "y": 120,
            "x": 632,
            "type": "host",
            "id": "H10"
        },
        {
            "ipv4": "10.0.0.13",
            "mac": "00:00:00:00:00:13",
            "y": 68,
            "x": 425,
            "type": "host",
            "id": "H12"
        }
    ],
    "links": [
        {
            "source": "H8",
            "capacity": 40,
            "weight": 2,
            "target": "S4",
            "ports": {
                "H8": 1,
                "S4": 1
            }
        },
        {
            "source": "H9",
            "capacity": 40,
            "weight": 2,
            "target": "S5",
            "ports": {
                "H9": 1,
                "S5": 1
            }
        },
        {
            "source": "H2",
            "capacity": 40,
            "weight": 2,
            "target": "S1",
            "ports": {
                "H2": 1,
                "S1": 1
            }
        },
        {
            "source": "H3",
            "capacity": 40,
            "weight": 2,
            "target": "S1",
            "ports": {
                "H3": 1,
                "S1": 2
            }
        },
        {
            "source": "S1",
            "capacity": 11,
            "weight": 20,
            "target": "S3",
            "ports": {
                "S1": 3,
                "S3": 1
            }
        },
        {
            "source": "S1",
            "capacity": 20,
            "weight": 7,
            "target": "S2",
            "ports": {
                "S1": 4,
                "S2": 1
            }
        },
        {
            "source": "S1",
            "capacity": 40,
            "weight": 2,
            "target": "H1",
            "ports": {
                "S1": 5,
                "H1": 1
            }
        },
        {
            "source": "H6",
            "capacity": 40,
            "weight": 2,
            "target": "S4",
            "ports": {
                "H6": 1,
                "S4": 2
            }
        },
        {
            "source": "H7",
            "capacity": 40,
            "weight": 2,
            "target": "S4",
            "ports": {
                "H7": 1,
                "S4": 3
            }
        },
        {
            "source": "S5",
            "capacity": 16,
            "weight": 10,
            "target": "S3",
            "ports": {
                "S5": 2,
                "S3": 2
            }
        },
        {
            "source": "S5",
            "capacity": 40,
            "weight": 2,
            "target": "H10",
            "ports": {
                "S5": 3,
                "H10": 1
            }
        },
        {
            "source": "S5",
            "capacity": 19,
            "weight": 27,
            "target": "S4",
            "ports": {
                "S5": 4,
                "S4": 4
            }
        },
        {
            "source": "S4",
            "capacity": 15,
            "weight": 17,
            "target": "S2",
            "ports": {
                "S4": 5,
                "S2": 2
            }
        },
        {
            "source": "S3",
            "capacity": 40,
            "weight": 2,
            "target": "H11",
            "ports": {
                "S3": 3,
                "H11": 1
            }
        },
        {
            "source": "S3",
            "capacity": 40,
            "weight": 2,
            "target": "H12",
            "ports": {
                "S3": 4,
                "H12": 1
            }
        },
        {
            "source": "S3",
            "capacity": 22,
            "weight": 23,
            "target": "S2",
            "ports": {
                "S3": 5,
                "S2": 3
            }
        },
        {
            "source": "H4",
            "capacity": 40,
            "weight": 2,
            "target": "S2",
            "ports": {
                "H4": 1,
                "S2": 4
            }
        },
        {
            "source": "S2",
            "capacity": 40,
            "weight": 2,
            "target": "H5",
            "ports": {
                "S2": 5,
                "H5": 1
            }
        }
    ]
}
// reveal.js plugins