Skip to content

Will fail on large files #6

Open
@Emerentius

Description

@Emerentius

The code below loads both files completely into memory before comparing. For files that differ early that can be quite wasteful and it can also result in out of memory errors.

// file a is guaranteed to exist...
let a_text = read_to_vec(a)?;

// but file b is not. If we have any kind of error when loading
// it up, that's a positive result, not an actual error.
let b_text = match read_to_vec(b) {
    Ok(contents) => contents,
    Err(_) => return Ok(true),
};

if a_text != b_text {
    return Ok(true);
}

I'd propose something like

let a_reader = BufReader::new( File::open(a)? );
let b_reader = match File::open(a) {
    Ok(f) => BufReader::new(f),
    Err(_) => return Ok(true),
}

if a_reader.bytes().ne( b_reader.bytes() ) {
    return Ok(true)
}

for similar functionality. The above won't compile when I'm reading the docs right, because the Bytes iterator returns Result<u8, io::Error> and io::Error doesn't implement PartialEq but one should be able to manually iterate in lockstep like so

let mut a_bytes = a_reader.bytes();
let mut b_bytes = b_reader.bytes();
loop {
    match (a_bytes().next(), b_bytes().next()) {
        (Some(Ok(a)), Some(Ok(b))) => if a != b { return Ok(true) },
        (None, None) => break,
        _ => return Ok(true),
    }
}

Now, you're also treating any kind of error as if that would indicate a difference in the folders but there's so much that can go wrong in the world that you can't really make that decision. It's certainly true for FileNotFound but what about some network storage that loses connection for example?
I don't know how much of that ugly mess you're willing to deal with of course.

Note: All code untested. I'm not at a full fledged PC at the moment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions