Add Json Schema Test Suite

[ci skip]

.gitignore

@@ -1,7 +1,3 @@
-/bin/*
-!/bin/data
-!/bin/encodings
-!/bin/jsonchecker
 /build
 /doc/html
 /doc/doxygen_*.db

bin/jsonschema/.gitignore

@@ -0,0 +1 @@
+TODO

bin/jsonschema/.travis.yml

@@ -0,0 +1,4 @@
+language: python
+python: "2.7"
+install: pip install jsonschema
+script: bin/jsonschema_suite check

bin/jsonschema/LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2012 Julian Berman
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

bin/jsonschema/README.md

@@ -0,0 +1,89 @@
+JSON Schema Test Suite [![Build Status](https://travis-ci.org/json-schema/JSON-Schema-Test-Suite.png?branch=develop)](https://travis-ci.org/json-schema/JSON-Schema-Test-Suite)
+======================
+
+This repository contains a set of JSON objects that implementors of JSON Schema
+validation libraries can use to test their validators.
+
+It is meant to be language agnostic and should require only a JSON parser.
+
+The conversion of the JSON objects into tests within your test framework of
+choice is still the job of the validator implementor.
+
+Structure of a Test
+-------------------
+
+If you're going to use this suite, you need to know how tests are laid out. The
+tests are contained in the `tests` directory at the root of this repository.
+
+Inside that directory is a subdirectory for each draft or version of the
+schema. We'll use `draft3` as an example.
+
+If you look inside the draft directory, there are a number of `.json` files,
+which logically group a set of test cases together. Often the grouping is by
+property under test, but not always, especially within optional test files
+(discussed below).
+
+Inside each `.json` file is a single array containing objects. It's easiest to
+illustrate the structure of these with an example:
+
+```json
+    {
+        "description": "the description of the test case",
+        "schema": {"the schema that should" : "be validated against"},
+        "tests": [
+            {
+                "description": "a specific test of a valid instance",
+                "data": "the instance",
+                "valid": true
+            },
+            {
+                "description": "another specific test this time, invalid",
+                "data": 15,
+                "valid": false
+            }
+        ]
+    }
+```
+
+So a description, a schema, and some tests, where tests is an array containing
+one or more objects with descriptions, data, and a boolean indicating whether
+they should be valid or invalid.
+
+Coverage
+--------
+
+Draft 3 and 4 should have full coverage. If you see anything missing or think
+there is a useful test missing, please send a pull request or open an issue.
+
+Who Uses the Test Suite
+-----------------------
+
+This suite is being used by:
+
+  * [json-schema-validator (Java)](https://github.com/fge/json-schema-validator)
+  * [jsonschema (python)](https://github.com/Julian/jsonschema)
+  * [aeson-schema (haskell)](https://github.com/timjb/aeson-schema)
+  * [direct-schema (javascript)](https://github.com/IreneKnapp/direct-schema)
+  * [jsonschema (javascript)](https://github.com/tdegrunt/jsonschema)
+  * [JaySchema (javascript)](https://github.com/natesilva/jayschema)
+  * [z-schema (javascript)](https://github.com/zaggino/z-schema)
+  * [jassi (javascript)](https://github.com/iclanzan/jassi)
+  * [json-schema-valid (javascript)](https://github.com/ericgj/json-schema-valid)
+  * [jesse (Erlang)](https://github.com/klarna/jesse)
+  * [json-schema (PHP)](https://github.com/justinrainbow/json-schema)
+  * [gojsonschema (Go)](https://github.com/sigu-399/gojsonschema) 
+  * [json_schema (Dart)](https://github.com/patefacio/json_schema) 
+  * [tv4 (JavaScript)](https://github.com/geraintluff/tv4)
+  * [Jsonary (JavaScript)](https://github.com/jsonary-js/jsonary)
+
+If you use it as well, please fork and send a pull request adding yourself to
+the list :).
+
+Contributing
+------------
+
+If you see something missing or incorrect, a pull request is most welcome!
+
+There are some sanity checks in place for testing the test suite. You can run
+them with `bin/jsonschema_suite check`. They will be run automatically by
+[Travis CI](https://travis-ci.org/) as well.

bin/jsonschema/bin/jsonschema_suite

@@ -0,0 +1,283 @@
+#! /usr/bin/env python
+from __future__ import print_function
+import sys
+import textwrap
+
+try:
+    import argparse
+except ImportError:
+    print(textwrap.dedent("""
+        The argparse library could not be imported. jsonschema_suite requires
+        either Python 2.7 or for you to install argparse. You can do so by
+        running `pip install argparse`, `easy_install argparse` or by
+        downloading argparse and running `python2.6 setup.py install`.
+
+        See https://pypi.python.org/pypi/argparse for details.
+    """.strip("\n")))
+    sys.exit(1)
+
+import errno
+import fnmatch
+import json
+import os
+import random
+import shutil
+import unittest
+import warnings
+
+if getattr(unittest, "skipIf", None) is None:
+    unittest.skipIf = lambda cond, msg : lambda fn : fn
+
+try:
+    import jsonschema
+except ImportError:
+    jsonschema = None
+else:
+    validators = getattr(
+        jsonschema.validators, "validators", jsonschema.validators
+    )
+
+
+ROOT_DIR = os.path.join(
+    os.path.dirname(__file__), os.pardir).rstrip("__pycache__")
+SUITE_ROOT_DIR = os.path.join(ROOT_DIR, "tests")
+
+REMOTES = {
+    "integer.json": {"type": "integer"},
+    "subSchemas.json": {
+        "integer": {"type": "integer"},
+        "refToInteger": {"$ref": "#/integer"},
+    },
+    "folder/folderInteger.json": {"type": "integer"}
+}
+REMOTES_DIR = os.path.join(ROOT_DIR, "remotes")
+
+TESTSUITE_SCHEMA = {
+    "$schema": "http://json-schema.org/draft-03/schema#",
+    "type": "array",
+    "items": {
+        "type": "object",
+        "properties": {
+            "description": {"type": "string", "required": True},
+            "schema": {"required": True},
+            "tests": {
+                "type": "array",
+                "items": {
+                    "type": "object",
+                    "properties": {
+                        "description": {"type": "string", "required": True},
+                        "data": {"required": True},
+                        "valid": {"type": "boolean", "required": True}
+                    },
+                    "additionalProperties": False
+                },
+                "minItems": 1
+            }
+        },
+        "additionalProperties": False,
+        "minItems": 1
+    }
+}
+
+
+def files(paths):
+    for path in paths:
+        with open(path) as test_file:
+            yield json.load(test_file)
+
+
+def groups(paths):
+    for test_file in files(paths):
+        for group in test_file:
+            yield group
+
+
+def cases(paths):
+    for test_group in groups(paths):
+        for test in test_group["tests"]:
+            test["schema"] = test_group["schema"]
+            yield test
+
+
+def collect(root_dir):
+    for root, dirs, files in os.walk(root_dir):
+        for filename in fnmatch.filter(files, "*.json"):
+            yield os.path.join(root, filename)
+
+
+class SanityTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        print("Looking for tests in %s" % SUITE_ROOT_DIR)
+        cls.test_files = list(collect(SUITE_ROOT_DIR))
+        print("Found %s test files" % len(cls.test_files))
+        assert cls.test_files, "Didn't find the test files!"
+
+    def test_all_files_are_valid_json(self):
+        for path in self.test_files:
+            with open(path) as test_file:
+                try:
+                    json.load(test_file)
+                except ValueError as error:
+                    self.fail("%s contains invalid JSON (%s)" % (path, error))
+
+    def test_all_descriptions_have_reasonable_length(self):
+        for case in cases(self.test_files):
+            descript = case["description"]
+            self.assertLess(
+                len(descript),
+                60,
+                "%r is too long! (keep it to less than 60 chars)" % (descript,)
+            )
+
+    def test_all_descriptions_are_unique(self):
+        for group in groups(self.test_files):
+            descriptions = set(test["description"] for test in group["tests"])
+            self.assertEqual(
+                len(descriptions),
+                len(group["tests"]),
+                "%r contains a duplicate description" % (group,)
+            )
+
+    @unittest.skipIf(jsonschema is None, "Validation library not present!")
+    def test_all_schemas_are_valid(self):
+        for schema in os.listdir(SUITE_ROOT_DIR):
+            schema_validator = validators.get(schema)
+            if schema_validator is not None:
+                test_files = collect(os.path.join(SUITE_ROOT_DIR, schema))
+                for case in cases(test_files):
+                    try:
+                        schema_validator.check_schema(case["schema"])
+                    except jsonschema.SchemaError as error:
+                        self.fail("%s contains an invalid schema (%s)" %
+                                  (case, error))
+            else:
+                warnings.warn("No schema validator for %s" % schema)
+
+    @unittest.skipIf(jsonschema is None, "Validation library not present!")
+    def test_suites_are_valid(self):
+        validator = jsonschema.Draft3Validator(TESTSUITE_SCHEMA)
+        for tests in files(self.test_files):
+            try:
+                validator.validate(tests)
+            except jsonschema.ValidationError as error:
+                self.fail(str(error))
+
+    def test_remote_schemas_are_updated(self):
+        for url, schema in REMOTES.items():
+            filepath = os.path.join(REMOTES_DIR, url)
+            with open(filepath) as schema_file:
+                self.assertEqual(json.load(schema_file), schema)
+
+
+def main(arguments):
+    if arguments.command == "check":
+        suite = unittest.TestLoader().loadTestsFromTestCase(SanityTests)
+        result = unittest.TextTestRunner(verbosity=2).run(suite)
+        sys.exit(not result.wasSuccessful())
+    elif arguments.command == "flatten":
+        selected_cases = [case for case in cases(collect(arguments.version))]
+
+        if arguments.randomize:
+            random.shuffle(selected_cases)
+
+        json.dump(selected_cases, sys.stdout, indent=4, sort_keys=True)
+    elif arguments.command == "remotes":
+        json.dump(REMOTES, sys.stdout, indent=4, sort_keys=True)
+    elif arguments.command == "dump_remotes":
+        if arguments.update:
+            shutil.rmtree(arguments.out_dir, ignore_errors=True)
+
+        try:
+            os.makedirs(arguments.out_dir)
+        except OSError as e:
+            if e.errno == errno.EEXIST:
+                print("%s already exists. Aborting." % arguments.out_dir)
+                sys.exit(1)
+            raise
+
+        for url, schema in REMOTES.items():
+            filepath = os.path.join(arguments.out_dir, url)
+
+            try:
+                os.makedirs(os.path.dirname(filepath))
+            except OSError as e:
+                if e.errno != errno.EEXIST:
+                    raise
+
+            with open(filepath, "wb") as out_file:
+                json.dump(schema, out_file, indent=4, sort_keys=True)
+    elif arguments.command == "serve":
+        try:
+            from flask import Flask, jsonify
+        except ImportError:
+            print(textwrap.dedent("""
+                The Flask library is required to serve the remote schemas.
+
+                You can install it by running `pip install Flask`.
+
+                Alternatively, see the `jsonschema_suite remotes` or
+                `jsonschema_suite dump_remotes` commands to create static files
+                that can be served with your own web server.
+            """.strip("\n")))
+            sys.exit(1)
+
+        app = Flask(__name__)
+
+        @app.route("/<path:path>")
+        def serve_path(path):
+            if path in REMOTES:
+                return jsonify(REMOTES[path])
+            return "Document does not exist.", 404
+
+        app.run(port=1234)
+
+
+parser = argparse.ArgumentParser(
+    description="JSON Schema Test Suite utilities",
+)
+subparsers = parser.add_subparsers(help="utility commands", dest="command")
+
+check = subparsers.add_parser("check", help="Sanity check the test suite.")
+
+flatten = subparsers.add_parser(
+    "flatten",
+    help="Output a flattened file containing a selected version's test cases."
+)
+flatten.add_argument(
+    "--randomize",
+    action="store_true",
+    help="Randomize the order of the outputted cases.",
+)
+flatten.add_argument(
+    "version", help="The directory containing the version to output",
+)
+
+remotes = subparsers.add_parser(
+    "remotes",
+    help="Output the expected URLs and their associated schemas for remote "
+         "ref tests as a JSON object."
+)
+
+dump_remotes = subparsers.add_parser(
+    "dump_remotes", help="Dump the remote ref schemas into a file tree",
+)
+dump_remotes.add_argument(
+    "--update",
+    action="store_true",
+    help="Update the remotes in an existing directory.",
+)
+dump_remotes.add_argument(
+    "--out-dir",
+    default=REMOTES_DIR,
+    type=os.path.abspath,
+    help="The output directory to create as the root of the file tree",
+)
+
+serve = subparsers.add_parser(
+    "serve",
+    help="Start a webserver to serve schemas used by remote ref tests."
+)
+
+if __name__ == "__main__":
+    main(parser.parse_args())

include/rapidjson/schema.h

@@ -18,7 +18,7 @@
 #include "document.h"
 #include <cmath> // HUGE_VAL, fmod
 
-#if !defined(RAPIDJSON_SCHEMA_USE_STDREGEX) && __cplusplus >=201103L
+#if !defined(RAPIDJSON_SCHEMA_USE_STDREGEX) && (__cplusplus >=201103L || (defined(_MSC_VER) && _MSC_VER >= 1800))
 #define RAPIDJSON_SCHEMA_USE_STDREGEX 1
 #endif
 
@@ -1225,6 +1225,18 @@ inline BaseSchema<Encoding>* CreateSchema(const ValueType& value, const ValueTyp
     else                                      return 0;
 }
 
+template <typename Encoding, typename ValueType>
+inline BaseSchema<Encoding>* CreateSchema(const ValueType& value, SchemaType type) {
+    if      (type == kNullSchemaType   ) return new NullSchema<Encoding>(value);
+    else if (type == kBooleanSchemaType) return new BooleanSchema<Encoding>(value);
+    else if (type == kObjectSchemaType ) return new ObjectSchema<Encoding>(value);
+    else if (type == kArraySchemaType  ) return new ArraySchema<Encoding>(value);
+    else if (type == kStringSchemaType ) return new StringSchema<Encoding>(value);
+    else if (type == kIntegerSchemaType) return new IntegerSchema<Encoding>(value);
+    else if (type == kNumberSchemaType ) return new NumberSchema<Encoding>(value);
+    else                                return 0;
+}
+
 template <typename Encoding, typename ValueType>
 inline BaseSchema<Encoding>* CreateSchema(const ValueType& value) {
     if (!value.IsObject())
@@ -1232,9 +1244,44 @@ inline BaseSchema<Encoding>* CreateSchema(const ValueType& value) {
 
     typename ValueType::ConstMemberIterator typeItr = value.FindMember("type");
 
-    if (typeItr == value.MemberEnd())  return new TypelessSchema<Encoding>(value);
+    if (typeItr == value.MemberEnd()) {
-    else if (typeItr->value.IsArray()) return new MultiTypeSchema<Encoding>(value, typeItr->value);
+        // Detect type with existing properties
-    else                               return CreateSchema<Encoding, ValueType>(value, typeItr->value);
+        struct PropertyMap {
+            const char* name;
+            SchemaType type;
+        };
+        static const PropertyMap kPropertyMap[] = {
+            { "additional", kArraySchemaType },
+            { "additionalProperties", kObjectSchemaType },
+            { "dependencies", kObjectSchemaType },
+            { "exclusiveMinimum", kNumberSchemaType },
+            { "exclusiveMaximum", kNumberSchemaType },
+            { "items", kArraySchemaType },
+            { "minimum", kNumberSchemaType },
+            { "minItems", kArraySchemaType },
+            { "minLength", kStringSchemaType },
+            { "minProperties", kObjectSchemaType },
+            { "maximum", kNumberSchemaType },
+            { "maxItems", kArraySchemaType },
+            { "maxLength", kStringSchemaType },
+            { "maxProperties", kObjectSchemaType },
+            { "multipleOf", kNumberSchemaType },
+            { "pattern", kStringSchemaType },
+            { "patternProperties", kObjectSchemaType },
+            { "properties", kObjectSchemaType },
+            { "required", kObjectSchemaType },
+        };
+
+        for (size_t i = 0; i < sizeof(kPropertyMap) / sizeof(kPropertyMap[0]); i++)
+            if (value.HasMember(kPropertyMap[i].name))
+                return CreateSchema<Encoding, ValueType>(value, kPropertyMap[i].type);
+        
+        return new TypelessSchema<Encoding>(value);
+    }
+    else if (typeItr->value.IsArray())
+        return new MultiTypeSchema<Encoding>(value, typeItr->value);
+    else
+        return CreateSchema<Encoding, ValueType>(value, typeItr->value);
 }
 
 template <typename Encoding, typename Allocator = MemoryPoolAllocator<> >

test/unittest/schematest.cpp

@@ -27,14 +27,8 @@ using namespace rapidjson;
     /*printf("\n%s\n", json);*/\
     d.Parse(json);\
     EXPECT_FALSE(d.HasParseError());\
-    if (expected) {\
+    EXPECT_TRUE(expected == d.Accept(validator));\
-        EXPECT_TRUE(d.Accept(validator));\
+    EXPECT_TRUE(expected == validator.IsValid());\
-        EXPECT_TRUE(validator.IsValid());\
-    }\
-    else {\
-        EXPECT_FALSE(d.Accept(validator));\
-        EXPECT_FALSE(validator.IsValid()); \
-    }\
 }
 
 TEST(SchemaValidator, Typeless) {
@@ -613,3 +607,111 @@ TEST(SchemaValidator, AllOf_Nested) {
     VALIDATE(s, "\"too long\"", false);
     VALIDATE(s, "123", false);
 }
+
+static char* ReadFile(const char* filename, size_t& length) {
+    const char *paths[] = {
+        "jsonschema/tests/draft4/%s",
+        "bin/jsonschema/tests/draft4/%s",
+        "../bin/jsonschema/tests/draft4/%s",
+        "../../bin/jsonschema/tests/draft4/%s",
+        "../../../bin/jsonschema/tests/draft4/%s"
+    };
+    char buffer[1024];
+    FILE *fp = 0;
+    for (size_t i = 0; i < sizeof(paths) / sizeof(paths[0]); i++) {
+        sprintf(buffer, paths[i], filename);
+        fp = fopen(buffer, "rb");
+        if (fp)
+            break;
+    }
+
+    if (!fp)
+        return 0;
+
+    fseek(fp, 0, SEEK_END);
+    length = (size_t)ftell(fp);
+    fseek(fp, 0, SEEK_SET);
+    char* json = (char*)malloc(length + 1);
+    size_t readLength = fread(json, 1, length, fp);
+    json[readLength] = '\0';
+    fclose(fp);
+    return json;
+}
+
+
+TEST(SchemaValidator, TestSuite) {
+    const char* filenames[] = {
+        "additionalItems.json",
+        "additionalProperties.json",
+        "allOf.json",
+        "anyOf.json",
+        "definitions.json",
+        "dependencies.json",
+        "enum.json",
+        "items.json",
+        "maximum.json",
+        "maxItems.json",
+        "maxLength.json",
+        "maxProperties.json",
+        "minimum.json",
+        "minItems.json",
+        "minLength.json",
+        "minProperties.json",
+        "multipleOf.json",
+        "not.json",
+        "oneOf.json",
+        "pattern.json",
+        "patternProperties.json",
+        "properties.json",
+        "ref.json",
+        "refRemote.json",
+        "required.json",
+        "type.json",
+        "uniqueItems.json"
+    };
+
+    unsigned testCount = 0;
+    unsigned passCount = 0;
+
+    for (size_t i = 0; i < sizeof(filenames) / sizeof(filenames[0]); i++) {
+        const char* filename = filenames[i];
+        size_t length;
+        char* json = ReadFile(filename, length);
+        if (!json) {
+            printf("json test suite file %s not found", filename);
+            ADD_FAILURE();
+        }
+        else {
+            Document d;
+            d.Parse(json);
+            if (d.HasParseError()) {
+                printf("json test suite file %s has parse error", filename);
+                ADD_FAILURE();
+            }
+            else {
+                for (Value::ConstValueIterator schemaItr = d.Begin(); schemaItr != d.End(); ++schemaItr) {
+                    Schema schema((*schemaItr)["schema"]);
+                    SchemaValidator validator(schema);
+                    const Value& tests = (*schemaItr)["tests"];
+                    for (Value::ConstValueIterator testItr = tests.Begin(); testItr != tests.End(); ++testItr) {
+                        testCount++;
+                        const Value& data = (*testItr)["data"];
+                        bool expected = (*testItr)["valid"].GetBool();
+                        const char* description = (*testItr)["description"].GetString();
+                        validator.Reset();
+                        bool actual = data.Accept(validator);
+                        if (expected != actual) {
+                            char buffer[256];
+                            sprintf(buffer, "%s \"%s\"", filename, description);
+                            GTEST_NONFATAL_FAILURE_(buffer);
+                        }
+                        else
+                            passCount++;
+                    }
+                }
+            }
+        }
+        free(json);
+    }
+    printf("%d / %d passed (%2d%%)\n", passCount, testCount, passCount * 100 / testCount);
+}