Skip to content

Re-thinking: Support deserializing key-value config data #322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tracked by #376
matthiasbeyer opened this issue Apr 23, 2022 · 3 comments
Open
Tracked by #376

Re-thinking: Support deserializing key-value config data #322

matthiasbeyer opened this issue Apr 23, 2022 · 3 comments

Comments

@matthiasbeyer
Copy link
Member

config-rs should be able to support key-value configuration data of the following types:

  • Bool
  • Integer
  • Float
  • String
  • List of "value"
  • Key-Value Mapping of "String" to "Value"

Something like

#[derive(serde::Deserialize)]
enum Value {
    Bool(bool),
    Int(i64),
    Float(f64),
    Text(String),
    List(Vec<Value>),
    Map(HashMap<String, Value>),
}

Should be the basis of all operations. This data format can be deserialized from different sources and be used for the "layering" functionality later.
It might be even possible to implement in a non-owning way.

We should keep the currently supported formats as supported formats in the future.

That entails:

  • Toml
  • Json
  • INI
  • Yaml
  • Ron
  • Json5
@xinslu
Copy link

xinslu commented Jun 25, 2022

I'd love to work on this, I've previously worked on an interpreter is Rust before, that had internal type system. I feel like I could port some of my code from there.

@belltoy
Copy link

belltoy commented Jun 26, 2022

I am interested in this thinking. A typical scenario is that loading from a backend key-value config service and needs to be merged to the 'layered' local configs. My previous solution was mapping those key-value string typed configs to an INI format, then later deserializing them to a structured type.

There are problems I am thinking about.

How to define the typed value, especially the List type of value and the Map type.

The List can be defined on the key or on the value. For example, we can define a comma-separated list value, but that's easy to do when deserializing. And another problem is this comma-separated value is hard work with other typed formats. If defined the List on the key, it should be like:

# key-value
foo.bar.[1] = 42
foo.bar.[2] = 43
fun_a.enabled = true
# Yaml
foo:
  bar:
      - 42
      - 43
fun_a:
  enabled: true
# env
# maybe like this
PREFIX_FOO_BAR_[1] = 42
PREFIX_FOO_BAR_[2] = 43
PREFIX_FUN_A_ENABLED = true
#[derive(Deserialize)]
pub struct Config {
    foo: Foo,

    fun_a: FunConfig,
}

#[derive(Deserialize)]
pub struct Foo {
    bar: Vec<u32>,
}

#[derive(Deserialize)]
pub struct FunConfig {
    enabled: bool,
}

This means it needs a structured key syntax definition. Something like this will be compatible with the other typed format (JSON, Yaml, etc). Furthermore, this seems like an extension of the Environment type.

@matthiasbeyer
Copy link
Member Author

Um... I am not sure we're talking about the same thing here.

The intention of the issue is that we do not deserialize to an internal data store, but to the normal data types that each format exposes (think serde_json::Value and toml::Value) and store all these as layers. The mentioned data type (from the code snippet) was to visualize what data types we support (all that are supported in the mentioned file formats). And we might implement this in a way that we do not own the data, to not copy data around when there is no need.

As far as I understood, you're talking about writing an own data format, but that was never the intention! 🤔

@matthiasbeyer matthiasbeyer mentioned this issue Sep 17, 2022
15 tasks
@epage epage removed the re-think label Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants