Transferring Files Over TCP With Bash
There are times when I have Ethernet working on an embedded GNU/Linux board, but no tool available to actually transfer files. Latest example is the Yocto build for an i.MX evaluation board from NXP that I just built. The system lives on the SD card, so adding a package would mean to reconnect the SD card to my development machine, copy the file(s) and put it back into the board. And I am just too lazy for that. The EVK has an IP address on my local network, so there must be an easy way to transfer a file with standard tools, right?
BASH To The Rescue
Having BASH available on the target, we can use redirection from a TCP
port without any additional helper. All that is needed on the other
side is a quick way to offer a single file when connecting to a
specific port. Turns out Netcat is such a tool and I usually have it
installed on my systems. So this one-liner starts nc listening to a
TCP port and will send the whole file to a connecting client and then
quit (host has IPv4 address 192.168.1.2):
dzu@krikkit:...imx-yocto-6.6.3-1.1.0/build-imx8mnlpddr4evk$ cat tmp/deploy/deb/armv8a/dropbear_2022.83-r0_arm64.deb \
| nc -q 1 -l 192.168.1.2 23234
With this, we can receive the file on the target and install it easily:
root@imx8mn-lpddr4-evk:~# cat < /dev/tcp/192.168.1.2/23234 > /tmp/new.deb
root@imx8mn-lpddr4-evk:~# dpkg -i /tmp/new.deb
We can then start the ssh service with systemctl start sshd and from
here on we are again on familiar grounds.
Serial Only
Writing the first paragraph reminded me that even without Ethernet there are multiple ways to transfer a file to an attached GNU/Linux board. One of the most important is when you have a serial connection only
Plain Text And (Almost) No Tools
It is a little bit more difficult to transfer files over serial only
as the communication is not safe against transmission errors. If we
disregard this, we could try transferring plain text files only with
cat. Let's start cat on the remote end and redirect all input to a file:
root@imx8mn-lpddr4-evk:~# cat > /tmp/rx-data
Then we close the serial connection, by entering "C-\ q" in kermit.
Back at our local machine, we create a simple text file and output
that to our serial port. We then reconnect with kermit, press "C-d"
to signal EOF to cat and inspect the output:
Closing /dev/ttyUSB2...OK
dzu@krikkit:~$ cat > /tmp/tx-data
line1
line2
line3
dzu@krikkit:~$ cat /tmp/tx-data > /dev/ttyUSB2
dzu@krikkit:~$ kermit -l /dev/ttyUSB2 -b 115200 -c
Connecting to /dev/ttyUSB2, speed 115200
Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------
root@imx8mn-lpddr4-evk:~# cat /tmp/rxdata
line1
line2
line3
root@imx8mn-lpddr4-evk:~#
Almost! There is something going on as the line feeds get duplicated.
Let's check the stty settings of the remote machine and our serial
port:
root@imx8mn-lpddr4-evk:~# stty -a
speed 115200 baud; rows 34; columns 110; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>;
start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon ixoff -iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon -iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc
root@imx8mn-lpddr4-evk:~#
Closing /dev/ttyUSB2...OK
dzu@krikkit:~$ stty -F /dev/ttyUSB2 -a
speed 115200 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>;
start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff -iuclc -ixany -imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten -echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc
dzu@krikkit:~$
There is a lot listed here, but the culprit is the onlcr setting on
the host and the corresponding icrnl on the remote end. Sending a
line feed will output lf and cr to go to the next line and return
to the first character. On the remote machine the cr will again be
translated into an lf and thus two line feeds will be transmitted!
Fixing this is very easy. We just need to fix the setting on our local machine:
dzu@krikkit:~$ stty -F /dev/ttyUSB2 -onlcr
dzu@krikkit:~$
If we repeat the "start cat on remote end, switch to local machine and cat test data" dance, things now look fine:
dzu@krikkit:~$ kermit -l /dev/ttyUSB2 -b 115200 -c
Connecting to /dev/ttyUSB2, speed 115200
Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------
root@imx8mn-lpddr4-evk:~# cat > /tmp/rx-data
Closing /dev/ttyUSB2...OK
dzu@krikkit:~$ cat /tmp/tx-data > /dev/ttyUSB2
dzu@krikkit:~$ kermit -l /dev/ttyUSB2 -b 115200 -c
Connecting to /dev/ttyUSB2, speed 115200
Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------
root@imx8mn-lpddr4-evk:~# cat /tmp/rx-data
line1
line2
line3
root@imx8mn-lpddr4-evk:~#
Binary Data
Trying this approach with binary data will fail, as several control
characters and especially EOF (0xd hexadecimal) will be interpreted
by the remote serial port instead of written to the data file. One
way to fix this would be to convert the binary data into pure text
files (i.e. UUENCODE, BASE64 or similar), but this usually also
requires extra tools on the remote end, so I will ignore this for now.
Instead we head for another option which will also add error
correction in the transmission. Such protocols were established when
the serial connection was all there was, i.e. in the days of modem
communication. One of the oldest is the XMODEM protocol. It is
implemented in many chat programs and command line utilities like rx
from the lrzsz package. If you have rx available on the host and
the target, kermit can be used nicely to transfer a file. It will
call sx and rx for you on both ends. This time we only press "C-\
c" to get to kermits command prompt and start the transfer from there:
root@imx8mn-lpddr4-evk:~#
(Back at krikkit)
----------------------------------------------------
(~/) C-Kermit>send /protocol:xmodem /etc/motd motd
Sende /etc/motd, 2 Blöcke:Starten Sie nun Ihr XMODEM-Empfangsprogramm.
Bytes gesendet: 384 BPS:359
Übertragung abgeschlossen
(~/) C-Kermit>c
Connecting to /dev/ttyUSB1, speed 115200
Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------
4-evk:~# ls
motd
root@imx8mn-lpddr4-evk:~#
A quick test showed that minicom also supports x/y/z-modem
transfers, but gtkterm does not. Oh well.
Summary
Transferring files has a very long history going back to dial up lines and sometimes the tools from that era are still very helpful.
Comments
Comments powered by Disqus