Skip to content

Commit 58c64c0

Browse files
rossbowenrubensworks
rossbowen
and
rubensworks
authored
Add RDFa datasource (#142)
* Add RDFa datasource * Polish RDFa datasource Co-authored-by: Ruben Taelman <[email protected]>
1 parent c2898b2 commit 58c64c0

File tree

18 files changed

+473
-0
lines changed

18 files changed

+473
-0
lines changed

Diff for: ‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ config/*.json
1313

1414
# Ignore dev environment files
1515
.idea
16+
.devcontainer

Diff for: ‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ The following modules are available:
114114
* [`@ldf/feature-webid`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/feature-webid): Feature that enables authenticated requests from clients with WebID.
115115
* [`@ldf/datasource-hdt`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-hdt): Datasource that allows HDT files to be loaded.
116116
* [`@ldf/datasource-jsonld`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-jsonld): Datasource that allows JSON-LD files to be loaded.
117+
* [`@ldf/datasource-rdfa`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-rdfa): Datasource that allows RDFa files to be loaded.
117118
* [`@ldf/datasource-n3`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-n3): Datasource that allows [N-Quads](https://www.w3.org/TR/n-quads/), [N-Triples](https://www.w3.org/TR/n-triples/), [Trig](https://www.w3.org/TR/trig/) and [Turtle](https://www.w3.org/TR/turtle/) files to be loaded.
118119
* [`@ldf/datasource-sparql`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-sparql): Datasource that allows SPARQL endpoints to be used as a data proxy.
119120
* [`@ldf/datasource-composite`](https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-composite): Datasource that delegates queries to an sequence of other datasources.

Diff for: ‎greenkeeper.json

+1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
"packages/datasource-composite/package.json",
1818
"packages/preset-qpf/package.json",
1919
"packages/datasource-jsonld/package.json",
20+
"packages/datasource-rdfa/package.json",
2021
"packages/feature-qpf/package.json",
2122
"packages/feature-summary/package.json"
2223
]

Diff for: ‎packages/datasource-rdfa/README.md

+57
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Linked Data Fragments Server - RDFa Datasource
2+
<img src="http://linkeddatafragments.org/images/logo.svg" width="200" align="right" alt="" />
3+
4+
[![npm version](https://badge.fury.io/js/%40ldf%2Fdatasource-rdfa.svg)](https://www.npmjs.com/package/@ldf/datasource-rdfa)
5+
6+
This module contains a RDFa datasource for the [Linked Data Fragments server](https://github.com/LinkedDataFragments/Server.js).
7+
It allows HTML files containing RDFa to be loaded.
8+
9+
_This package is a [Linked Data Fragments Server module](https://github.com/LinkedDataFragments/Server.js/)._
10+
11+
## Usage in `@ldf/server`
12+
13+
This package exposes the following config entries:
14+
* `RdfaDatasource`: A RDFa datasource that requires at least one `file` field. _Should be used as `@type` value._
15+
16+
Example:
17+
```json
18+
{
19+
"@context": "https://linkedsoftwaredependencies.org/bundles/npm/@ldf/server/^3.0.0/components/context.jsonld",
20+
"@id": "urn:ldf-server:my",
21+
"import": "preset-qpf:config-defaults.json",
22+
23+
"datasources": [
24+
{
25+
"@id": "urn:ldf-server:myRdfaDatasource",
26+
"@type": "RdfaDatasource",
27+
"datasourceTitle": "My RDFa HTML file",
28+
"description": "My dataset with a RDFa back-end",
29+
"datasourcePath": "myrdfa",
30+
"file": "path/to/file.html"
31+
}
32+
]
33+
}
34+
```
35+
36+
## Usage in other packages
37+
38+
When this module is used in a package other than `@ldf/server`,
39+
then the JSON-LD context `https://linkedsoftwaredependencies.org/contexts/@ldf/datasource-rdfa.jsonld` must be imported.
40+
41+
For example:
42+
```
43+
{
44+
"@context": [
45+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/core/^3.0.0/components/context.jsonld",
46+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/preset-qpf/^3.0.0/components/context.jsonld",
47+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-rdfa/^3.0.0/components/context.jsonld",
48+
],
49+
// Same as above...
50+
}
51+
```
52+
53+
## License
54+
The Linked Data Fragments server is written by [Ruben Verborgh](https://ruben.verborgh.org/), Miel Vander Sande, [Ruben Taelman](https://www.rubensworks.net/) and colleagues.
55+
56+
This code is copyrighted by [Ghent University – imec](http://idlab.ugent.be/)
57+
and released under the [MIT license](http://opensource.org/licenses/MIT).

Diff for: ‎packages/datasource-rdfa/components/Datasource/Rdfa.jsonld

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"@context": [
3+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/core/^3.0.0/components/context.jsonld",
4+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-rdfa/^3.0.0/components/context.jsonld"
5+
],
6+
"@id": "npmd:@ldf/datasource-rdfa",
7+
"components": [
8+
{
9+
"@id": "ldfdr:Datasource/Rdfa",
10+
"@type": "Class",
11+
"extends": "ldfc:Datasource/Memory",
12+
"requireElement": "datasources.RdfaDatasource",
13+
"comment": "An RdfaDatasource fetches data from an RDFa HTML document",
14+
"constructorArguments": {
15+
"extends": "ldfc:Datasource/Memory#constructorArgumentsObject"
16+
}
17+
}
18+
]
19+
}

Diff for: ‎packages/datasource-rdfa/components/components.jsonld

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"@context": "https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-rdfa/^3.0.0/components/context.jsonld",
3+
"@id": "npmd:@ldf/datasource-rdfa",
4+
"@type": "Module",
5+
"requireName": "@ldf/datasource-rdfa",
6+
"import": [
7+
"files-ldfdr:components/Datasource/Rdfa.jsonld"
8+
]
9+
}

Diff for: ‎packages/datasource-rdfa/components/context.jsonld

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"@context": [
3+
"https://linkedsoftwaredependencies.org/bundles/npm/componentsjs/^4.0.0/components/context.jsonld",
4+
{
5+
"npmd": "https://linkedsoftwaredependencies.org/bundles/npm/",
6+
"ldfdr": "npmd:@ldf/datasource-rdfa/",
7+
"files-ldfdr": "ldfdr:^3.0.0/",
8+
"RdfaDatasource": "ldfdr:Datasource/Rdfa"
9+
}
10+
]
11+
}

Diff for: ‎packages/datasource-rdfa/config/config-example.json

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"@context": "https://linkedsoftwaredependencies.org/bundles/npm/@ldf/server/^3.0.0/components/context.jsonld",
3+
"@id": "urn:ldf-server:my",
4+
"import": "preset-qpf:config-defaults.json",
5+
6+
"datasources": [
7+
{
8+
"@id": "urn:ldf-server:myRdfaDatasource",
9+
"@type": "RdfaDatasource",
10+
"datasourceTitle": "My RDFa HTML file",
11+
"description": "My dataset with a RDFa back-end",
12+
"datasourcePath": "myrdfa",
13+
"file": "path/to/file.html"
14+
}
15+
]
16+
}

Diff for: ‎packages/datasource-rdfa/index.js

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/*! @license MIT ©2015-2016 Ruben Verborgh, Ghent University - imec */
2+
/* Exports of the components of this package */
3+
4+
module.exports = {
5+
datasources: {
6+
RdfaDatasource: require('./lib/datasources/RdfaDatasource'),
7+
},
8+
};

Diff for: ‎packages/datasource-rdfa/lib/datasources/RdfaDatasource.js

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
/*! @license MIT ©2014-2016 Ruben Verborgh, Ghent University - imec */
2+
/* An RdfaDatasource fetches data from a JSON-LD document. */
3+
4+
let MemoryDatasource = require('@ldf/core').datasources.MemoryDatasource,
5+
RdfaParser = require('rdfa-streaming-parser').RdfaParser;
6+
7+
let ACCEPT = 'text/html;q=1.0,application/xhtml+xml;q=0.7';
8+
9+
// Creates a new RdfaDatasource
10+
class RdfaDatasource extends MemoryDatasource {
11+
constructor(options) {
12+
super(options);
13+
this._url = options && (options.url || options.file);
14+
}
15+
16+
// Retrieves all quads from the document
17+
_getAllQuads(addQuad, done) {
18+
let document = this._fetch({ url: this._url, headers: { accept: ACCEPT } });
19+
new RdfaParser({ baseIRI: this._url, dataFactory: this.dataFactory })
20+
.import(document)
21+
.on('error', done)
22+
.on('data', addQuad)
23+
.on('end', done);
24+
}
25+
}
26+
27+
module.exports = RdfaDatasource;

Diff for: ‎packages/datasource-rdfa/package.json

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"name": "@ldf/datasource-rdfa",
3+
"description": "Linked Data Fragments Server - RDFa Datasource",
4+
"version": "3.2.1",
5+
"lsd:module": true,
6+
"license": "MIT",
7+
"main": "index.js",
8+
"publishConfig": {
9+
"access": "public"
10+
},
11+
"files": [
12+
"components",
13+
"lib/**/*.js",
14+
"index.js"
15+
],
16+
"repository": "https://github.com/LinkedDataFragments/Server.js/tree/master/packages/datasource-rdfa",
17+
"bugs": {
18+
"url": "https://github.com/LinkedDataFragments/Server.js/issues"
19+
},
20+
"scripts": {
21+
"test": "nyc mocha",
22+
"lint": "eslint bin/* lib test"
23+
},
24+
"dependencies": {
25+
"rdf-string": "^1.3.1",
26+
"rdfa-streaming-parser": "^1.5.0"
27+
},
28+
"peerDependencies": {
29+
"@ldf/core": "^3.0.0"
30+
},
31+
"devDependencies": {
32+
"@ldf/core": "^3.2.1"
33+
}
34+
}

Diff for: ‎packages/datasource-rdfa/test/.eslintrc

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
globals: {
3+
describe: true,
4+
it: true,
5+
before: true,
6+
after: true,
7+
beforeEach: true,
8+
afterEach: true,
9+
expect: true,
10+
sinon: true,
11+
test: true,
12+
},
13+
14+
rules: {
15+
new-cap: 0, // test constructors as regular functions
16+
max-nested-callbacks: 0, // Mocha works with deeply nested callbacks
17+
},
18+
}

Diff for: ‎packages/datasource-rdfa/test/datasources/RdfaDatasource-test.js

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
/*! @license MIT ©2014-2016 Ruben Verborgh, Ghent University - imec */
2+
let RdfaDatasource = require('../../').datasources.RdfaDatasource;
3+
4+
let Datasource = require('@ldf/core').datasources.Datasource,
5+
path = require('path'),
6+
dataFactory = require('n3').DataFactory;
7+
8+
let exampleRdfaUrl = 'file://' + path.join(__dirname, '../../../../test/assets/test.html');
9+
10+
describe('RdfaDatasource', () => {
11+
describe('The RdfaDatasource module', () => {
12+
it('should be a function', () => {
13+
RdfaDatasource.should.be.a('function');
14+
});
15+
16+
it('should be a RdfaDatasource constructor', (done) => {
17+
let instance = new RdfaDatasource({ dataFactory, url: exampleRdfaUrl });
18+
instance.should.be.an.instanceof(RdfaDatasource);
19+
instance.close(done);
20+
});
21+
22+
it('should create Datasource objects', (done) => {
23+
let instance = new RdfaDatasource({ dataFactory, url: exampleRdfaUrl });
24+
instance.should.be.an.instanceof(Datasource);
25+
instance.close(done);
26+
});
27+
});
28+
29+
describe('A RdfaDatasource instance for an example RDFa HTML file', () => {
30+
let datasource = new RdfaDatasource({ dataFactory, url: exampleRdfaUrl });
31+
datasource.initialize();
32+
after((done) => { datasource.close(done); });
33+
34+
itShouldExecute(datasource,
35+
'the empty query',
36+
{ features: { triplePattern: true } },
37+
129, 129);
38+
39+
itShouldExecute(datasource,
40+
'the empty query with a limit',
41+
{ limit: 10, features: { triplePattern: true, limit: true } },
42+
10, 129);
43+
44+
itShouldExecute(datasource,
45+
'the empty query with an offset',
46+
{ offset: 10, features: { triplePattern: true, offset: true } },
47+
119, 129);
48+
49+
itShouldExecute(datasource,
50+
'a query for an existing subject',
51+
{ subject: dataFactory.namedNode('http://example.org/s1'), limit: 10, features: { triplePattern: true, limit: true } },
52+
10, 100);
53+
54+
itShouldExecute(datasource,
55+
'a query for a non-existing subject',
56+
{ subject: dataFactory.namedNode('http://example.org/p1'), limit: 10, features: { triplePattern: true, limit: true } },
57+
0, 0);
58+
59+
itShouldExecute(datasource,
60+
'a query for an existing predicate',
61+
{ predicate: dataFactory.namedNode('http://example.org/p1'), limit: 10, features: { triplePattern: true, limit: true } },
62+
10, 110);
63+
64+
itShouldExecute(datasource,
65+
'a query for a non-existing predicate',
66+
{ predicate: dataFactory.namedNode('http://example.org/s1'), limit: 10, features: { triplePattern: true, limit: true } },
67+
0, 0);
68+
69+
itShouldExecute(datasource,
70+
'a query for an existing object',
71+
{ object: dataFactory.namedNode('http://example.org/o001'), limit: 10, features: { triplePattern: true, limit: true } },
72+
3, 3);
73+
74+
itShouldExecute(datasource,
75+
'a query for a non-existing object',
76+
{ object: dataFactory.namedNode('http://example.org/s1'), limit: 10, features: { triplePattern: true, limit: true } },
77+
0, 0);
78+
});
79+
});
80+
81+
function itShouldExecute(datasource, name, query, expectedResultsCount, expectedTotalCount) {
82+
describe('executing ' + name, () => {
83+
let resultsCount = 0, totalCount;
84+
before((done) => {
85+
let result = datasource.select(query);
86+
result.getProperty('metadata', (metadata) => { totalCount = metadata.totalCount; });
87+
result.on('data', (triple) => { resultsCount++; });
88+
result.on('end', done);
89+
});
90+
91+
it('should return the expected number of triples', () => {
92+
expect(resultsCount).to.equal(expectedResultsCount);
93+
});
94+
95+
it('should emit the expected total number of triples', () => {
96+
expect(totalCount).to.equal(expectedTotalCount);
97+
});
98+
});
99+
}

Diff for: ‎packages/datasource-rdfa/test/mocha.opts

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
--require ../../test/test-setup
2+
--recursive
3+
--timeout 500

Diff for: ‎packages/server/components/context.jsonld

+1
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-hdt/^3.0.0/components/context.jsonld",
66
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-jsonld/^3.0.0/components/context.jsonld",
77
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-n3/^3.0.0/components/context.jsonld",
8+
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-rdfa/^3.0.0/components/context.jsonld",
89
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/datasource-sparql/^3.0.0/components/context.jsonld",
910
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/feature-memento/^3.0.0/components/context.jsonld",
1011
"https://linkedsoftwaredependencies.org/bundles/npm/@ldf/feature-qpf/^3.0.0/components/context.jsonld",

Diff for: ‎packages/server/package.json

+1
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
"@ldf/datasource-composite": "^3.2.1",
3232
"@ldf/datasource-hdt": "^3.2.1",
3333
"@ldf/datasource-jsonld": "^3.2.1",
34+
"@ldf/datasource-rdfa": "^3.0.0",
3435
"@ldf/datasource-n3": "^3.2.1",
3536
"@ldf/datasource-sparql": "^3.2.1",
3637
"@ldf/feature-memento": "^3.2.1",

0 commit comments

Comments
 (0)