Skip to content

Support reading configurations from files #8338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 46 commits into from
Feb 24, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
c512b78
Initial commit
paullegranddc Feb 3, 2025
dc449e0
Initial tests for StableConfigSource
mtoffl01 Feb 4, 2025
6d72f81
Add @Override for stableconfig get and configorigin methods
mtoffl01 Feb 4, 2025
92c81fd
apply github code qual suggestions
mtoffl01 Feb 4, 2025
635ba8a
passing initial tests
mtoffl01 Feb 5, 2025
d5bf963
support config_id
mtoffl01 Feb 5, 2025
9f03e04
Additional tests for invalid files
mtoffl01 Feb 5, 2025
06d572d
Introduce StableConfigSource to the list of ConfigProvider sources
mtoffl01 Feb 6, 2025
a03562c
add comments to current tests
mtoffl01 Feb 7, 2025
ac7b387
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 7, 2025
908a72d
fix typo
mtoffl01 Feb 7, 2025
abfc789
Merge branch 'mtoff/stable_config' of github.com:DataDog/dd-trace-jav…
mtoffl01 Feb 7, 2025
38355fe
Refactor: Introduce StableConfigSource sub-class, StableConfig
mtoffl01 Feb 10, 2025
aebebbe
nits: docs and cleanup
mtoffl01 Feb 10, 2025
90b52e9
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 10, 2025
cdd8df8
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 10, 2025
1fed1da
add doc for getStableConfig
mtoffl01 Feb 10, 2025
9f2b252
Merge branch 'mtoff/stable_config' of github.com:DataDog/dd-trace-jav…
mtoffl01 Feb 10, 2025
2da99f2
Merge branch 'master' into mtoff/stable_config
mtoffl01 Feb 10, 2025
ae6036b
Optimize: lazily load configuration information from file, cache results
mtoffl01 Feb 12, 2025
e3d6c80
nits: variable renaming, comments to describe variables
mtoffl01 Feb 12, 2025
a7bf5ba
nit: move comment above variable
mtoffl01 Feb 12, 2025
0c8ef6a
fix typo in managed file path
mtoffl01 Feb 13, 2025
049da8b
Initial StableConfigParser implementation and testing
mtoffl01 Feb 13, 2025
39fa0e2
Finalize StableConfigParser and tests
mtoffl01 Feb 13, 2025
53dec92
User parser in StableconfigSource; no more lazy loading
mtoffl01 Feb 13, 2025
96881d0
Add new logs for testing
mtoffl01 Feb 13, 2025
09e22a7
remove snakeyaml from dependency tree in internal-api
mtoffl01 Feb 13, 2025
519c443
Fix class initialization order
mcculls Feb 14, 2025
9a519e1
update parser test to use raw file input as opposed to map
mtoffl01 Feb 15, 2025
a9f968f
make ParserTest handle unexpected input format as well
mtoffl01 Feb 18, 2025
7c93f15
Revert fileretries stuff
mtoffl01 Feb 18, 2025
36a7e8a
Change managed path to reflect system tests
mtoffl01 Feb 18, 2025
9598e11
remove superfluous comments/logs used for testing
mtoffl01 Feb 18, 2025
bf9efad
Merge branch 'master' into mtoff/stable_config
mtoffl01 Feb 18, 2025
a34a546
snakeyaml is only used for testing now
mcculls Feb 19, 2025
a3be88d
Register StableConfigSource with native-image since we need it at bui…
mcculls Feb 19, 2025
99cb07b
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 20, 2025
cf99c4f
Address PR comments
mtoffl01 Feb 20, 2025
3d1a1d8
Change javadoc on getStableConfig function
mtoffl01 Feb 20, 2025
41fd3bc
Update dd-java-agent/agent-bootstrap/src/main/java/datadog/trace/boot…
mtoffl01 Feb 20, 2025
9dd51fe
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 20, 2025
f143632
Update internal-api/src/main/java/datadog/trace/bootstrap/config/prov…
mtoffl01 Feb 20, 2025
ea8d9f7
Apply PR suggestions round 2
mtoffl01 Feb 20, 2025
4366cab
Apply PR suggestions round 3
mtoffl01 Feb 21, 2025
eb7618b
Merge branch 'master' into mtoff/stable_config
mtoffl01 Feb 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions internal-api/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ dependencies {
// it contains annotations that are also present in the instrumented application classes
api "com.datadoghq:dd-javac-plugin-client:0.2.2"

implementation 'org.yaml:snakeyaml:2.0'

testImplementation project(":utils:test-utils")
testImplementation("org.assertj:assertj-core:3.20.2")
testImplementation libs.bundles.junit5
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ public enum ConfigOrigin {
REMOTE("remote_config"),
/** configurations that are set through JVM properties */
JVM_PROP("jvm_prop"),
/** configuration read in the stable config file, managed by users */
USER_STABLE_CONFIG("user_stable_config"),
/** configuration read in the stable config file, managed by fleet */
MANAGED_STABLE_CONFIG("managed_stable_config"),
/** set when the user has not set any configuration for the key (defaults to a value) */
DEFAULT("default");

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package datadog.trace.bootstrap.config.provider;

import static datadog.trace.util.Strings.propertyNameToEnvironmentVariableName;

import datadog.trace.api.ConfigOrigin;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.SafeConstructor;

public final class StableConfigSource extends ConfigProvider.Source {
static final String USER_STABLE_CONFIG_PATH = "/etc/datadog-agent/application_monitoring.yaml";
static final String MANAGED_STABLE_CONFIG_PATH =
"/etc/datadog-agent/managed/datadog-apm-libraries/stable/application_monitoring.yaml ";
private static final Logger log = LoggerFactory.getLogger(StableConfigSource.class);

final ConfigOrigin fileOrigin;
HashMap<String, Object> configuration;

StableConfigSource(String file, ConfigOrigin origin) {
this.fileOrigin = origin;
try {
configuration = parseStableConfig(file);
} catch (Exception e) {
configuration = new HashMap<>();
}
}

private static HashMap<String, Object> parseStableConfig(String filePath) throws IOException {
HashMap<String, Object> config = new HashMap<>();

// Check if the file exists
File file = new File(filePath);
if (!file.exists()) {
log.error("Stable configuration file does not exist at the specified path: {}", filePath);
return config; // Exit early or take other action as necessary
}

Yaml yaml = new Yaml(new SafeConstructor(new LoaderOptions()));
InputStream input = Files.newInputStream(new File(filePath).toPath());
HashMap<String, Object> data = yaml.load(input);

Object apmConfig = data.get("apm_configuration_default");
if (apmConfig instanceof HashMap<?, ?>) {
HashMap<?, ?> tempConfig = (HashMap<?, ?>) apmConfig;
for (Map.Entry<?, ?> entry : tempConfig.entrySet()) {
if (entry.getKey() instanceof String && entry.getValue() != null) {
String key = String.valueOf(entry.getKey());
Object value = entry.getValue();
config.put(key, value);
} else {
log.debug("Configuration {} in unexpected format", entry.getKey());
}
}
} else {
// do something
log.debug("File {} in unexpected format", filePath);
}
return config;
};

public String get(String key) {
return (String) configuration.get(propertyNameToEnvironmentVariableName(key));
}

public Set<String> getKeys() {
return this.configuration.keySet();
}

public ConfigOrigin origin() {
return fileOrigin;
}

private static class StableConfig {
private String config_id;
private HashMap<String, Object> apm_configuration_default;

private void setApmConfigurationDefault(HashMap<String, Object> m) {
apm_configuration_default = m;
}

private void setConfigId(String i) {
config_id = i;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package datadog.trace.bootstrap.config.provider

import datadog.trace.api.ConfigOrigin
import datadog.trace.test.util.DDSpecification
import org.yaml.snakeyaml.DumperOptions
import org.yaml.snakeyaml.Yaml

class StableConfigSourceTest extends DDSpecification {

def "test file doesn't exist"() {
setup:
StableConfigSource config = new StableConfigSource(StableConfigSource.USER_STABLE_CONFIG_PATH, ConfigOrigin.USER_STABLE_CONFIG)

expect:
config.getKeys().size() == 0
// How to check that the "file does not exist" error was logged?
}

def "test valid file"() {
// test empty file
when:
def path = StableConfigSource.USER_STABLE_CONFIG_PATH
try {
File file = new File(path)
file.createNewFile()
} catch (IOException e) {
// fail fast?
System.out.println("Error creating file")
e.printStackTrace()
}
StableConfigSource config = new StableConfigSource(path, ConfigOrigin.USER_STABLE_CONFIG)

then:
config.getKeys().size() == 0

// test populated file
// when:
// def key1 = "dd_first_key"
// def val1 = "dd_first_val"
// def key2 = "dd_second_key"
// def val2 = "dd_second_val"
// // Create the map that will be used to populate the config file
// Map<String, Object> data = new HashMap<>();
// data.put("apm_configuration_default", new HashMap<String, Object>() {{
// put(key1, val1);
// put(key2, val2);
// }})
//
// DumperOptions options = new DumperOptions();
// options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK);
//
// // Prepare to write the data map to the file in yaml format
// Yaml yaml = new Yaml(options);
//
// try (FileWriter writer = new FileWriter(path)) {
// yaml.dump(data, writer);
// } catch (IOException e) {
// System.err.println("Error writing to file: " + e.getMessage());
// // fail fast?
// }
//
// then:
// StableConfigSource config2 = new StableConfigSource(StableConfigSource.USER_STABLE_CONFIG_PATH, ConfigOrigin.USER_STABLE_CONFIG);
// config2.getKeys().size() == 2
// config2.get(key1) == val1
// config2.get(key2) == val2
}
}
Loading