https://github.com/keithasaurus/koda-validate

Skip to content Toggle navigation
 
Sign up

  * Product
      +  
        Actions
        Automate any workflow
      +  
        Packages
        Host and manage packages
      +  
        Security
        Find and fix vulnerabilities
      +  
        Codespaces
        Instant dev environments
      +  
        Copilot
        Write better code with AI
      +  
        Code review
        Manage code changes
      +  
        Issues
        Plan and track work
      +  
        Discussions
        Collaborate outside of code
      + Explore
      + All features
      + Documentation
      + GitHub Skills
      + Blog
  * Solutions
      + By Plan
      + Enterprise
      + Teams
      + Compare all
      + By Solution
      + CI/CD & Automation
      + DevOps
      + DevSecOps
      + Case Studies
      + Customer Stories
      + Resources
  * Open Source
      +  
        GitHub Sponsors
        Fund open source developers
      +  
        The ReadME Project
        GitHub community articles
      + Repositories
      + Topics
      + Trending
      + Collections
  * Pricing

[                    ] 

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this user All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in
Sign up
{{ message }}
keithasaurus / koda-validate Public

  * Notifications
  * Fork 0
  * Star 14

Typesafe Validation

License

MIT license
14 stars 0 forks
Star
Notifications

  * Code
  * Issues 0
  * Pull requests 0
  * Actions
  * Projects 0
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Security
  * Insights

keithasaurus/koda-validate

This commit does not belong to any branch on this repository, and may
belong to a fork outside of the repository.
main
Switch branches/tags
[                    ]
Branches Tags
Could not load branches
Nothing to show
{{ refName }} default View all branches
Could not load tags
Nothing to show
{{ refName }} default
View all tags

Name already in use

A tag already exists with the provided branch name. Many Git commands
accept both tag and branch names, so creating this branch may cause
unexpected behavior. Are you sure you want to create this branch?
Cancel Create
9 branches 0 tags
Code

  *  
    Clone
    HTTPS GitHub CLI
    [https://github.com/k]

    Use Git or checkout with SVN using the web URL.

    [gh repo clone keitha]

    Work fast with our official CLI. Learn more.

  * Open with GitHub Desktop
  * Download ZIP

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

@keithasaurus
keithasaurus typo
...
ba983ea Oct 25, 2022
typo
ba983ea

Git stats

  * 33 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.github
Dict codegen (#3)
Oct 24, 2022
codegen
1 zero (#4)
Oct 25, 2022
examples
1 zero (#4)
Oct 25, 2022
koda_validate
1 zero (#4)
Oct 25, 2022
tests
1 zero (#4)
Oct 25, 2022
.gitignore
remove .idea
Feb 5, 2022
.pre-commit-config.yaml
Dict codegen (#3)
Oct 24, 2022
LICENSE
Dict codegen (#3)
Oct 24, 2022
README.md
typo
Oct 25, 2022
poetry.lock
Dict codegen (#3)
Oct 24, 2022
pyproject.toml
1.0.0
Oct 25, 2022
setup.cfg
Dict codegen (#3)
Oct 24, 2022
View code
[                    ]
Koda Validate Install The Basics The (More) Basics Validation Errors
Validators, Predicates, and Extension Metadata Other Noteworthy
Validators and Utilities OneOf2 / OneOf3 Tuple2Validator /
Tuple3Validator Lazy MapValidator OptionalValidator maybe_key
is_dict_validator Limitations dict_validator has a max keys limit
dict_validator types may be hard to read / slow for your editor or
type-checker dict_validator's keys only allow for strings Your
Imagination Something's Missing Or Wrong

README.md

 Koda Validate

Typesafe, combinable validation. Python 3.8+

Koda Validate aims to make writing validators easier.

 Install

pip

pip install koda_validate

Poetry

poetry add koda_validate

 The Basics

from dataclasses import dataclass
from koda import Ok
from koda_validate import *


@dataclass
class Person:
    name: str
    age: int


person_validator = dict_validator(
    Person,
    key("name", StringValidator()),
    key("age", IntValidator())
)

result = person_validator({"name": "John Doe", "age": 30})
if isinstance(result, Ok):
    print(f"{result.val.name} is {result.val.age} years old")
else:
    print(result.val)

We could also nest person_validator, for instance, in a
ListValidator:

people_validator = ListValidator(person_validator)

...And nest that in a different validator (and so forth...):

@dataclass
class Group:
    name: str
    people: list[Person]


group_validator = dict_validator(
    Group,
    key("name", StringValidator()),
    key("people", people_validator),
)

data = {
    "name": "Arrested Development Characters",
    "people": [
        {"name": "George Bluth", "age": 70},
        {"name": "Michael Bluth", "age": 35}
    ]
}

assert group_validator(data) == Ok(
    Group(
        name='Arrested Development Characters',
        people=[
            Person(name='George Bluth', age=70),
            Person(name='Michael Bluth', age=35)
        ]
    )
)

Let's look at the dict_validator a bit closer. Its first argument can
be any Callable that accepts the values from each key below it -- in
the same order they are defined (the names of the keys and the
Callable arguments do not need to match). For person_validator, we
used a Person dataclass; for Group, we used a Group dataclass; but
that does not need to be the case. Because we can use any Callable
with matching types, this would also be valid:

from koda import Ok
from koda_validate import *


def reverse_person_args_tuple(a: str, b: int) -> tuple[int, str]:
    return b, a

person_validator_2 = dict_validator(
    reverse_person_args_tuple,
    key("name", StringValidator()),
    key("age", IntValidator()),
)

assert person_validator_2({"name": "John Doe", "age": 30}) == Ok((30, "John Doe"))

As you see, we have some flexibility in defining what we want to get
back from a validated dict_validator.

Another thing to note is that, so far, the results are all wrapped in
an Ok class. The other possibility -- when validation fails -- is
that an Err instance is returned, with whatever message or object a
given validator returns. We do not raise exceptions to express
validation failure in Koda Validate. Instead, validation is treated
as part of normal control flow.

Let's use some more features.

from dataclasses import dataclass
from koda import Err, Ok, Result
from koda_validate import *


@dataclass
class Employee:
    title: str
    name: str


def no_dwight_regional_manager(employee: Employee) -> Result[Employee, Serializable]:
    if (
        "schrute" in employee.name.lower()
        and employee.title.lower() == "assistant regional manager"
    ):
        return Err("Assistant TO THE Regional Manager!")
    else:
        return Ok(employee)


employee_validator = dict_validator(
    Employee,
    key("title", StringValidator(not_blank, MaxLength(100), preprocessors=[strip])),
    key("name", StringValidator(not_blank, preprocessors=[strip])),
    # After we've validated individual fields, we may want to validate them as a whole
    validate_object=no_dwight_regional_manager,
)


# The fields are valid but the object as a whole is not.
assert employee_validator(
    {
        "title": "Assistant Regional Manager",
        "name": "Dwight Schrute",
    }
) == Err("Assistant TO THE Regional Manager!")

Things to note about employee_validator:

  * we can add additional checks -- Predicates -- to validators (e.g.
    not_blank, MaxLength, etc.)
  * we can pre-process strings for formatting (after the type is
    determined, but before Predicate validators are run)
  * we have two stages of validation on dictionaries: first the keys,
    then the entire object, via validate_object
  * apparently we have a problem with someone named Dwight Schrute
    giving himself the wrong title

Note that everything we've seen is typesafe according to mypy -- with
strict settings, and without any plugins.

 The (More) Basics

We're are spending a lot of time discussing validating collections,
but Koda Validate works just as well on simple values.

from koda import Err, Ok
from koda_validate import ExactValidator, MinLength, StringValidator

min_length_3_validator = StringValidator(MinLength(4))
assert min_length_3_validator("good") == Ok("good")
assert min_length_3_validator("bad") == Err(["minimum allowed length is 4"])

exactly_5_validator = ExactValidator(5)

assert exactly_5_validator(5) == Ok(5)
assert exactly_5_validator("hmm") == Err(["expected exactly 5 (int)"])

Koda Validate is intended to be extendable enough to validate any
type of data.

 Validation Errors

As mentioned above, errors are returned as data as part of normal
control flow. The contents of all returned Errs from built-in
validators in Koda Validate are JSON/YAML serializable. (However,
should you build your own custom validators, there is no contract
enforcing that constraint.) Here are a few examples of the kinds of
errors you can expect to see out of the box.

from dataclasses import dataclass
from koda import Err, Maybe
from koda_validate import *

# Wrong type
assert StringValidator()(None) == Err(["expected a string"])

# All failing `Predicate`s are reported (not just the first)
str_choice_validator = StringValidator(MinLength(2), Choices({"abc", "yz"}))
assert str_choice_validator("") == Err(
    ["minimum allowed length is 2", "expected one of ['abc', 'yz']"]
)


@dataclass
class City:
    name: str
    region: Maybe[str]


city_validator = dict_validator(
    City,
    key("name", StringValidator(not_blank)),
    maybe_key("region", StringValidator(not_blank)),
)

# We use the key "__container__" for object-level errors
assert city_validator(None) == Err({"__container__": ["expected a dictionary"]})

# Missing keys are errors
assert city_validator({}) == Err({"name": ["key missing"]})

# Extra keys are also errors
assert city_validator(
    {"region": "California", "population": 510, "country": "USA"}
) == Err({"__container__": ["Received unknown keys. Only expected ['name', 'region']"]})


@dataclass
class Neighborhood:
    name: str
    city: City


neighborhood_validator = dict_validator(
    Neighborhood, key("name", StringValidator(not_blank)), key("city", city_validator)
)

# Errors are nested in predictable manner
assert neighborhood_validator({"name": "Bushwick", "city": {}}) == Err(
    {"city": {"name": ["key missing"]}}
)

If you have any concerns about being able to handle specific types of
key or object requirements, please see some of the other validators
and helpers below:

  * OneOf2 / OneOf3
  * MapValidator
  * OptionalValidator
  * maybe_key
  * is_dict_validator
  * Lazy

 Validators, Predicates, and Extension

Koda Validate's intention is to cover the bulk of common use cases
with its built-in tools. However, it is also meant to provide a
straightforward way to build for custom validation use-cases. Here
we'll provide a quick overview of how custom validation logic can be
implemented.

There are two kinds of Callables used for validation in Koda
Validate: Validators and Predicates. Validators can take an input of
one type and produce a valid result of another type. (While a
Validator has the capability to alter a value and/or type, whether it
does is entirely dependent on the given Validators requirements.)
Most commonly Validators accept type Any and validate that it
conforms to some type or data shape. As an example, we'll write a
simple Validator for floats here:

from typing import Any
from koda import Err, Ok, Result
from koda_validate.typedefs import Serializable, Validator


class SimpleFloatValidator(Validator[Any, float, Serializable]):
    def __call__(self, val: Any) -> Result[float, Serializable]:
        if isinstance(val, float):
            return Ok(val)
        else:
            return Err("expected a float")


float_validator = SimpleFloatValidator()
float_val = 5.5
assert float_validator(float_val) == Ok(float_val)
assert float_validator(5) == Err("expected a float")

What is this doing?

  * extending Validator, using the following types:
      + Any: any type of input can be passed in
      + float: if the data is valid, a value of type Ok[float] will
        be returned
      + Serializable: if it's invalid, a value of type Err
        [Serializable] will be returned
  * the __call__ method performs any kind of validation needed, so
    long as the input and output type signatures -- as determined by
    the Validator type parameters - are abided

We accept Any because the type of input may be unknown before
submitting to the Validator. After our validation in
SimpleFloatValidator succeeds, we know the type must be float.

This is all well and good, but we'll probably want to be able to
validate against values of the floats, such as min, max, or rough
equality checks. For this we use Predicates. This is what the
FloatValidator in Koda Validate looks like:

class FloatValidator(Validator[Any, float, Serializable]):
    def __init__(self, *predicates: Predicate[float, Serializable]) -> None:
        self.predicates = predicates

    def __call__(self, val: Any) -> Result[float, Serializable]:
        if isinstance(val, float):
            return accum_errors_serializable(val, self.predicates)
        else:
            return Err(["expected a float"])

Predicates are meant to validate the value of a known type -- as
opposed to validating at the type-level. For example, this is how you
might write and use a Predicate for approximate float equality:

import math
from dataclasses import dataclass
from koda import Err, Ok
from koda_validate import FloatValidator, Serializable, Predicate


@dataclass
class IsClose(Predicate[float, Serializable]):
    compare_to: float
    tolerance: float

    def is_valid(self, val: float) -> bool:
        return math.isclose(self.compare_to, val, abs_tol=self.tolerance)

    def err(self, val: float) -> Serializable:
        return f"expected a value within {self.tolerance} of {self.compare_to}"


# let's use it
close_to_validator = FloatValidator(IsClose(0.05, 0.02))
a = 0.06
assert close_to_validator(a) == Ok(a)
assert close_to_validator(0.01) == Err(["expected a value within 0.02 of 0.05"])

Notice that in Predicates we define is_valid and err methods, while
in Validators we define the entire __call__ method. This is because
the base Predicate class is constructed in such a way that we limit
how much it can actually do -- we don't want it to be able to alter
the value being validated. This turns out to be useful because it
allows us to proceed sequentially through an arbitrary amount of
Predicates of the same type in a given Validator. Only because of
this property can we be confident in our ability to return all
Predicate errors for a given Validator -- instead of having to exit
at the first failure.

 Metadata

Previously we said an aim of Koda Validate is to allow reuse of
validator metadata. Principally this is useful in generating
descriptions of the validator's constraints -- one example could be
generating an OpenAPI (or other) schema. Here we'll do something
simpler and use validator metadata to build a function which can
return plaintext descriptions of validators:

from typing import Any
from koda_validate import MaxLength, MinLength, Predicate, StringValidator, Validator


def describe_validator(validator: Validator[Any, Any, Any] | Predicate[Any, Any]) -> str:
    # use `isinstance(...)` in python <= 3.10
    match validator:
        case StringValidator(predicates):
            predicate_descriptions = [
                f"- {describe_validator(pred)}" for pred in predicates
            ]
            return "\n".join(["validates a string"] + predicate_descriptions)
        case MinLength(length):
            return f"minimum length {length}"
        case MaxLength(length):
            return f"maximum length {length}"
        # ...etc
        case _:
            raise TypeError(f"unhandled validator type. got {type(validator)}")


print(describe_validator(StringValidator()))
# validates a string
print(describe_validator(StringValidator(MinLength(5))))
# validates a string
# - minimum length 5
print(describe_validator(StringValidator(MinLength(3), MaxLength(8))))
# validates a string
# - minimum length 3
# - maximum length 8

All we're doing here, of course, is writing an interpreter. For the
sake of brevity this one is very simple, but it's straightforward to
extend the logic. This is easy to do because, while the validators
are Callables at their core, they are also classes that can easily be
inspected. (This ease of inspection is the primary reason we use
classes in Koda Validate.) Interpreters are the recommended way to
re-use validator metadata for non-validation purposes.

 Other Noteworthy Validators and Utilities

 OneOf2 / OneOf3

OneOfN validators are useful when you may have multiple valid shapes
of data.

from koda import First, Ok, Second

from koda_validate import ListValidator, OneOf2, StringValidator

string_or_list_string_validator = OneOf2(
    StringValidator(), ListValidator(StringValidator())
)

assert string_or_list_string_validator("ok") == Ok(First("ok"))
assert string_or_list_string_validator(["list", "of", "strings"]) == Ok(
    Second(["list", "of", "strings"])
)

 Tuple2Validator / Tuple3Validator

These Validators work on tuples as you might expect:

from koda import Ok
from koda_validate import IntValidator, StringValidator, Tuple2Validator

string_int_validator = Tuple2Validator(StringValidator(), IntValidator())

assert string_int_validator(("ok", 100)) == Ok(("ok", 100))

# also ok with lists
assert string_int_validator(["ok", 100]) == Ok(("ok", 100))

 Lazy

Lazy's main purpose is to allow for the use of recursion in
validation. An example use case of this might be replies in a comment
thread. This can be done with mutually recursive functions. For
simplicity, here's an example of parsing a kind of non-empty list.

from typing import Optional
from koda import Ok
from koda_validate import IntValidator, Lazy, OptionalValidator, Tuple2Validator

NonEmptyList = tuple[int, Optional["NonEmptyList"]]


def recur_non_empty_list() -> Tuple2Validator[int, Optional[NonEmptyList]]:
    return non_empty_list_validator


non_empty_list_validator = Tuple2Validator(
    IntValidator(),
    OptionalValidator(Lazy(recur_non_empty_list)),
)

assert non_empty_list_validator((1, (1, (2, (3, (5, None)))))) == Ok(
    (1, (1, (2, (3, (5, None)))))
)

 MapValidator

MapValidator allows us to validate dictionaries that are mappings of
one type to another type, where we don't need to be concerned about
individual keys or values:

from koda import Ok
from koda_validate import IntValidator, MapValidator, StringValidator

str_to_int_validator = MapValidator(StringValidator(), IntValidator())

assert str_to_int_validator({"a": 1, "b": 25, "xyz": 900}) == Ok(
    {"a": 1, "b": 25, "xyz": 900}
)

 OptionalValidator

OptionalValidator is very simple. It validates a value is either None
or passes another validator's rules.

from koda import Ok
from koda_validate import IntValidator, OptionalValidator

optional_int_validator = OptionalValidator(IntValidator())

assert optional_int_validator(5) == Ok(5)
assert optional_int_validator(None) == Ok(None)

 maybe_key

maybe_key allows for a key to be missing from a dictionary

from dataclasses import dataclass
from koda import Just, Maybe, Ok, nothing
from koda_validate import IntValidator, StringValidator, dict_validator, key, maybe_key


@dataclass
class Person:
    name: str
    age: Maybe[int]


person_validator = dict_validator(
    Person, key("name", StringValidator()), maybe_key("age", IntValidator())
)
assert person_validator({"name": "Bob"}) == Ok(Person("Bob", nothing))
assert person_validator({"name": "Bob", "age": 42}) == Ok(Person("Bob", Just(42)))

 is_dict_validator

A very simple validator that only validates that and object is a
dict. It doesn't do any validation against keys or values.

from koda import Ok, Err
from koda_validate.dictionary import is_dict_validator

assert is_dict_validator({}) == Ok({})
assert is_dict_validator(None) == Err({"__container__": ["expected a dictionary"]})
assert is_dict_validator({"a": 1, "b": 2, 5: "xyz"}) == Ok(
    {"a": 1, "b": 2, 5: "xyz"}
)

 Limitations

 dict_validator has a max keys limit

By default dict_validator can have a maximum of 20 keys. You can
change this by generating code and storing it in your project:

# allow up to 30 keys
python /path/to/koda-validate/codegen/generate.py /your/target/directory --num-keys 30

This limitation exists because computation starts to get expensive
for type checkers above a certain level, and it's not common to have
that many keys in a dict.

 dict_validator types may be hard to read / slow for your editor or
type-checker

dict_validator is a convenience function that delegates to different
Validators depending on the number of keys -- for example,
Dict2KeysValidator, Dict3KeysValidator, etc. These numbered
validators are limited to a specific number of keys and can be used
to mitigate such issues.

 dict_validator's keys only allow for strings

This should be resolved in a later release.

 Your Imagination



 Something's Missing Or Wrong

Open an issue on GitHub please!

About

Typesafe Validation

Resources

Readme

License

MIT license

Stars

14 stars

Watchers

1 watching

Forks

0 forks

Releases

No releases published

Packages 0

No packages published

Languages

  * Python 100.0%

Footer

 (c) 2022 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.