Skip to content

File base class needed for Arduino #4201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
PaulStoffregen opened this issue Jan 20, 2018 · 16 comments
Closed

File base class needed for Arduino #4201

PaulStoffregen opened this issue Jan 20, 2018 · 16 comments
Assignees
Milestone

Comments

@PaulStoffregen
Copy link

PaulStoffregen commented Jan 20, 2018

Arduino needs a File (and maybe a Dir) base class, similar to Stream, Print, Client & Server, so that libraries which actually use files can be given a hardware-neutral File object. Ideally, a library like a JPEG decoder would be able to use a File from SPIFFS or SD or Bridge or USB Host Shield or any other library which provides access to files.

As mentioned on #2281, here is a new issue to discuss how this might be achieved. Ideally, Arduino's core libraries for AVR, SAM, SAMD, and ESP8266 & ESP32, and Teensy, ChipKit, Photon & Energia could all use this same File base class, as we all currently do with Print & Stream, so that users of all our platforms can have File providing and File using libraries "just work". :-)

@tofrnr
Copy link

tofrnr commented Jan 20, 2018

I would appreciate more to have a FILE* base instead which is cstdio compatible (for SD file system and stream) and decline and deprecate all Arduino "File" bucklings for the future.

@tonton81
Copy link

+1 for File and Dir class :)

@PaulStoffregen
Copy link
Author

Just wanted to follow up regarding the idea to implement File more efficiently (mentioned on the Arduino dev mail list). Is this github issue ok to discuss, or should we use the Arduino dev list?

Even if virtual functions and polymorphism are used, we really do need a File (and maybe Dir) base class in Arduino, so libraries using File can work with any other library providing File.

@devyte
Copy link
Collaborator

devyte commented Jan 30, 2018

It is ok to discuss here. Actually, the ball is in my court atm, I have been asked to provide an outline for a templated design. I already have an ongoing discussion with a work collegue, and I have an idea of what it should look like.
About a base class, you're making the same mistake others make, by thinking that a base class would make File usable by others. In the template world, it is the interface that is important (I.e.: how is the class meant to be used), as opposed to who its ancestor is (what the class is).

@earlephilhower
Copy link
Collaborator

Love the idea of defining an interface, but Stream and Print are presently full classes, not templates. Would File being a template change the way folks would have to use it (vs. what they have to do today w/a Stream where they just subclass and implement the overrides)?

@PaulStoffregen
Copy link
Author

I hope you'll also consider whether templates really offer performance improvement over "devirtualization" by the compiler. On Teensy, we've found the gcc 5.4 does a good job of removing the vtable overhead when the constructors are defined with "constexpr".

@drmpf
Copy link

drmpf commented Jan 30, 2018 via email

@PaulStoffregen
Copy link
Author

Any progress or news to share on the alternative to polymorphism?

I don't want to be impatient. Arduino has used the same File class in SD & Bridge for years. A few weeks is not much time by comparison. But I am hoping we can move forward this month (February 2018) with something. Personally, I am ok with polymorphism & virtual functions if something better is not found.

@PaulStoffregen
Copy link
Author

Are you still working on the a non-polymorphism approach? If not, maybe it's time to start trying to move forward with the traditional Arduino polymorphism way?

@devyte
Copy link
Collaborator

devyte commented Feb 19, 2018

Yes, it hasn't been dropped. I had an extended trip, so there was a pause, but I'll be working on it again this week.
Please don't become impatient. I think this is the first chance for a full redesign, and the only reason the discussion has come up is because of the many different filesystem implementations out there. If this is adopted, it could set a precedent for improved design guidelines for other Arduino objects.
Let's do things right.

@PaulStoffregen
Copy link
Author

Any more progress on this?

While I want to be patient, soon I need to begin work on supporting USB memory sticks....

@PaulStoffregen
Copy link
Author

Two months have passed, or close to 3 if counting from the beginning of the conversation on Arduino's developer mail list on December 28, 2017, though the suggestion to wait for coming up with a template-based approach started on January 19 and this issue was opened on January 20.

Now we're at March 21. Will you ever show us this template idea?

I want to begin really working on porting libraries in April. Is that too soon? Am I being unreasonable?

@Misiu
Copy link

Misiu commented Apr 27, 2018

@PaulStoffregen @devyte would this solve problem with name collision described here: #3819

@PaulStoffregen
Copy link
Author

PaulStoffregen commented Apr 27, 2018

@Misiu - That only solves a small but important subset of the problem. You can include either of those in a sketch and use File. But you can't create a library that actually does something with File (eg, a JPEG decoder) that can accept any File. It's a good quick fix which allows many programs to compile with the existing libraries. But it's a poor long-term solution for the Arduino ecosystem.

earlephilhower referenced this issue May 17, 2018
…rn SSL (#4273)

BearSSL (https://www.bearssl.org) is a TLS(SSL) library written by
Thomas Pornin that is optimized for lower-memory embedded systems
like the ESP8266. It supports a wide variety of modern ciphers and
is unique in that it doesn't perform any memory allocations during
operation (which is the unfortunate bane of the current axTLS).

BearSSL is also absolutely focused on security and by default performs
all its security checks on x.509 certificates during the connection
phase (but if you want to be insecure and dangerous, that's possible
too).

While it does support unidirectional SSL buffers, like axTLS,
as implemented the ESP8266 wrappers only support bidirectional
buffers. These bidirectional buffers avoid deadlocks in protocols
which don't have well separated receive and transmit periods.

This patch adds several classes which allow connecting to TLS servers
using this library in almost the same way as axTLS:
BearSSL::WiFiClientSecure - WiFiClient that supports TLS
BearSSL::WiFiServerSecure - WiFiServer supporting TLS and client certs

It also introduces objects for PEM/DER encoded keys and certificates:
BearSSLX509List - x.509 Certificate (list) for general use
BearSSLPrivateKey - RSA or EC private key
BearSSLPublicKey - RSA or EC public key (i.e. from a public website)

Finally, it adds a Certificate Authority store object which lets
BearSSL access a set of trusted CA certificates on SPIFFS to allow it
to verify the identity of any remote site on the Internet, without
requiring RAM except for the single matching certificate.
CertStoreSPIFFSBearSSL - Certificate store utility

Client certificates are supported for the BearSSL::WiFiClientSecure, and
what's more the BearSSL::WiFiServerSecure can also *require* remote clients
to have a trusted certificate signed by a specific CA (or yourself with
self-signing CAs).

Maximum Fragment Length Negotiation probing and usage are supported, but
be aware that most sites on the Internet don't support it yet.  When
available, you can reduce the memory footprint of the SSL client or server
dramatically (i.e. down to 2-8KB vs. the ~22KB required for a full 16K
receive fragment and 512b send fragment).  You can also manually set a
smaller fragment size and guarantee at your protocol level all data will
fit within it.

Examples are included to show the usage of these new features.

axTLS has been moved to its own namespace, "axtls".  A default "using"
clause allows existing apps to run using axTLS without any changes.

The BearSSL::WiFi{client,server}Secure implements the axTLS
client/server API which lets many end user applications take advantage
of BearSSL with few or no changes.

The BearSSL static library used presently is stored at
https://github.com/earlephilhower/bearssl-esp8266 and can be built
using the standard ESP8266 toolchain.
@earlephilhower
Copy link
Collaborator

#5525 implements a compatible File/Filesystem interface for SPIFFS and SD(SDFS) (and LittleFS in #5511).

When that lands, I think this can be closed.

@earlephilhower earlephilhower self-assigned this Jan 22, 2019
@earlephilhower earlephilhower added this to the 2.6.0 milestone Jan 22, 2019
@earlephilhower
Copy link
Collaborator

Closing since we now have SD, SDFS, SPIFFS, (and soon LittleFS) all using a common, compatible File class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants