Skip to content

GzipFile BLFWriter negative seek error #1378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
j-c-cook opened this issue Aug 22, 2022 · 2 comments · Fixed by #1429
Closed

GzipFile BLFWriter negative seek error #1378

j-c-cook opened this issue Aug 22, 2022 · 2 comments · Fixed by #1429
Labels

Comments

@j-c-cook
Copy link
Contributor

j-c-cook commented Aug 22, 2022

Describe the bug

A GZipFile object implementation in cpython is unable to be sought negatively while in write mode. This is problematic for the BLFWriter. Upon initialization, the BLFWriter writes a default header to the file, in which the start and stop timestamps are not set. Once the BLFWriter is told to stop, the remaining data in the buffer is flushed to the file and then the header must be updated with the start and stop timestamps. This is where the GZipFile object has problems. The location of the writer must be moved to the 0'th position of the file. When in write mode, the offset position cannot be negatively sought.

To Reproduce

Socket CAN setup

Enable CAN interface:

sudo ip link add dev vcan0 type vcan && sudo ip link set up vcan0

Generate random CAN messages 1 millisecond apart:

cangen vcan0 -e -I 18EAFF26 -L i -v -v -v -g 1

python-can setup

Enable the Logger:

python -m can.logger -c vcan0 -i socketcan -b 250000 -f file.blf.gz

The issue can be seen with the SizedRotatingLogger as well.

python -m can.logger -c vcan0 -i socketcan -b 250000 -f file.blf.gz -s 150000

Expected behavior

A file.blf.gz should be created without error.

Additional context

OS and version: Kubuntu 22.04
Python version: 3.7.13
python-can version: pip install git+https://github.com/hardbyte/python-can@2da28c1a1c87776618a60218b0b97800cf2deb34
python-can interface/s (if applicable):

Traceback and logs

Logger trace back:

(pycan) jackcook@rollo:~$ python -m can.logger -c vcan0 -i socketcan -b 250000 -f file.blf.gz
Connected to SocketcanBus: socketcan channel 'vcan0'
Can Logger (Started on 2022-08-27 10:59:58.127917)
^CTraceback (most recent call last):
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/logger.py", line 268, in <module>
    main()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/logger.py", line 264, in main
    logger.stop()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/blf.py", line 589, in stop
    self.file.seek(0)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/gzip.py", line 371, in seek
    raise OSError('Negative seek in write mode')
OSError: Negative seek in write mode

SizedRotatingLogger trace back:

(pycan) jackcook@rollo:~$ python -m can.logger -c vcan0 -i socketcan -b 250000 -f file.blf.gz -s 150000
Connected to SocketcanBus: socketcan channel 'vcan0'
Can Logger (Started on 2022-08-27 11:01:30.452316)
Traceback (most recent call last):
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/logger.py", line 259, in main
    logger(msg)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/listener.py", line 43, in __call__
    self.on_message_received(msg)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/logger.py", line 206, in on_message_received
    self.do_rollover()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/logger.py", line 336, in do_rollover
    self.writer.stop()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/blf.py", line 589, in stop
    self.file.seek(0)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/gzip.py", line 371, in seek
    raise OSError('Negative seek in write mode')
OSError: Negative seek in write mode

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/logger.py", line 268, in <module>
    main()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/logger.py", line 264, in main
    logger.stop()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/logger.py", line 241, in stop
    self.writer.stop()
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/site-packages/can/io/blf.py", line 589, in stop
    self.file.seek(0)
  File "/home/jackcook/anaconda3/envs/pycan/lib/python3.7/gzip.py", line 371, in seek
    raise OSError('Negative seek in write mode')
OSError: Negative seek in write mode
@j-c-cook j-c-cook added the bug label Aug 22, 2022
@j-c-cook j-c-cook changed the title Fix the compressed BLFWriter and compressed SizedRotatingLogger rollover GzipFile BLFWriter negative seek error Aug 27, 2022
@christiansandberg
Copy link
Collaborator

Should be fixed somehow but it is also quite unnecessary to gzip a blf file since it already uses compression internally.

@j-c-cook
Copy link
Contributor Author

... it is also quite unnecessary to gzip a blf file since it already uses compression internally.

This is something that would be useful to quantify. I am using secure copy protocol, scp, to transfer CAN logs from a remote device to a server. I have ensured that the files being transferred are zipped due to some previous experiences (empiricism).
I have found in the past when transferring many files or even just one large file with scp that a zipped file is transported much faster. I am curious though now that you say this, in the case of blf versus blf.gz, what the percent difference in transfer time is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants