Re: Setting up NBD server / client - block nbd4: Other side returned error (22) issue
On Thu, Jul 22, 2021 at 01:41:00PM +0100, Simon Fernandez wrote:
> Hi folks,
> I’m trying setup a client that connect to a 500Gb ext4 disk on another machine
> running nbd server.
> Server: raspberry pi 3: uname -r 4.19.66+
> Client: raspberry pi 3: uname -r 5.10.17-v7+
>
> I’ve followed the information here [https://github.com/NetworkBlockDevice/nbd]
> and a lot of other sites but I think I’m doing something wrong on the server
> setup because I get this on the client:
>
> $ sudo mount /dev/nbd2 /home/pi/Vols/nbd1-recipes/
> mount: /home/pi/Vols/nbd1-recipes: can't read superblock on /dev/nbd2
>
> Also it appears to be listening on default port - NOT ones specified in config
> file.
>
> This is how I’ve set up the server & client, it’s probably something obvious to
> more experienced folk.
Wouter has answered this already, I just wanted to add a couple of
smaller points:
(1) You don't need to serve each export on a separate port. The NBD
protocol can serve multiple exports (disks) on the same port, with the
client requesting which one it wants.
(2) Use the 'nbdinfo' tool to examine what is being served by an NBD
server. It can query all sorts of information as described in the
manual:
https://libguestfs.org/nbdinfo.1.html
Rich.
> == rpi-C3 (192.168.1.14) server setup
> > sudo apt-get update
> > sudo apt-get install nbd-server
> > sudo nano /etc/nbd-server/config
> [generic]
> # If you want to run everything as root rather than the nbd user, you
> # may either say "root" in the two following lines, or remove them
> # altogether. Do not remove the [generic] section, however.
> user = nbd
> group = nbd
> includedir = /etc/nbd-server/conf.d
> allowlist = true
>
> # What follows are export definitions. You may create as much of them as
> # you want, but the section header has to be unique.
> [images]
> exportname = /home/pi/media/500G_FAITH/images
> port = 510029
> [recipes]
> exportname = /home/pi/media/500G_FAITH/recipes
> port = 510030
> [movie]
> exportname = /home/pi/media/500G_FAITH/movies
> port = 510031
> > sudo modprobe nbd # NOT nbd-server!
> > lsmod | grep nbd
> nbd 40960 0
> > sudo service nbd-server force-reload # if already running
>
> == client side
> > sudo apt-get install nbd-client # done
> > sudo modprobe nbd
> > nbd-client 192.168.1.14 -l
> Negotiation: ..
> images
> recipes
> movie
> > sudo nbd-client 192.168.1.14 -N recipes /dev/nbd2
> Negotiation: ..size = 0MB
> Connected /dev/nbd2
> > sudo nbd-client 192.168.1.14 -N movie /dev/nbd3
> Negotiation: ..size = 0MB
> Connected /dev/nbd3
> > sudo nbd-client 192.168.1.14 -N images /dev/nbd1
> Negotiation: ..size = 0MB
> Connected /dev/nbd1
>
> > dmesg | tail -n 10
> [ 1417.883364] Buffer I/O error on dev nbd1, logical block 0, async page read
> [ 1417.884614] block nbd1: Other side returned error (22)
> [ 1417.884652] blk_update_request: I/O error, dev nbd1, sector 0 op 0x0:(READ)
> flags 0x0 phys_seg 1 prio class 0
> [ 1417.884673] Buffer I/O error on dev nbd1, logical block 0, async page read
> [ 1417.885977] block nbd1: Other side returned error (22)
> [ 1417.886016] blk_update_request: I/O error, dev nbd1, sector 0 op 0x0:(READ)
> flags 0x0 phys_seg 1 prio class 0
> [ 1417.886036] Buffer I/O error on dev nbd1, logical block 0, async page read
> [ 1417.887276] block nbd1: Other side returned error (22)
> [ 1417.888492] block nbd1: Other side returned error (22)
> [ 1417.889751] block nbd1: Other side returned error (22)
> > sudo nbd-client 192.168.1.14 -d /dev/nbd1 # thought it might be something to
> do w/ nbd1
> > sudo nbd-client 192.168.1.14 -N images /dev/nbd4
> Negotiation: ..size = 0MB
> Connected /dev/nbd4
> > dmesg | tail -n 10
> [ 1848.831897] block nbd4: Other side returned error (22)
> [ 1848.833341] block nbd4: Other side returned error (22)
> [ 1848.834651] block nbd4: Other side returned error (22)
> [ 1848.836002] block nbd4: Other side returned error (22)
> > lsblk -f
> NAME FSTYPE LABEL UUID FSAVAIL
> FSUSE% MOUNTPOINT
> sda
>
> ├─sda1 vfat EFI 67E3-17ED
>
> └─sda2 hfsplus time_box_2018 f6427a3d-363b-3284-88c4-03ce3493aeff 115.5G
> 94% /media/pi/time_box_2018
> nbd2
>
> nbd3
>
> nbd4
>
> mmcblk0
>
> ├─mmcblk0p1 vfat boot 5DE4-665C 203.4M
> 19% /boot
> └─mmcblk0p2 ext4 rootfs 7295bbc3-bbc2-4267-9fa0-099e10ef5bf0 4.1G
> 37% /
> > dmesg | tail -n 10
> [ 1848.836002] block nbd4: Other side returned error (22)
> [ 1848.837430] block nbd4: Other side returned error (22)
> [ 1848.838860] block nbd4: Other side returned error (22)
> [ 1848.840224] block nbd4: Other side returned error (22)
> [ 1848.841691] block nbd4: Other side returned error (22)
> [ 1848.843077] block nbd4: Other side returned error (22)
> [ 1848.844415] block nbd4: Other side returned error (22)
> [ 2802.615037] block nbd3: Receive control failed (result -32)
> [ 2802.625871] block nbd2: Receive control failed (result -32)
> [ 2802.648444] block nbd4: Receive control failed (result -32)
> > ls /home/pi/Vols/
> nbd1-recipes/ rpi-C2-nbd-500G/ tbx2018/ time_box_2018/
> > sudo mount /dev/nbd2 /home/pi/Vols/nbd1-recipes/
> mount: /home/pi/Vols/nbd1-recipes: can't read superblock on /dev/nbd2.
> > sudo parted -l
> Model: Seagate BUP Slim SL (scsi)
> Disk /dev/sda: 2000GB
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
> Disk Flags:
>
> Number Start End Size File system Name Flags
> 1 20.5kB 210MB 210MB fat32 EFI System Partition boot, esp
> 2 210MB 2000GB 2000GB hfs+ time_box_2018
>
>
> Error: /dev/nbd3: unrecognised disk label
> Model: Unknown (unknown)
> Disk /dev/nbd3: 4096B
> Sector size (logical/physical): 512B/512B
> Partition Table: unknown
> Disk Flags:
>
> Error: /dev/nbd4: unrecognised disk label
> Model: Unknown (unknown)
> Disk /dev/nbd4: 4096B
> Sector size (logical/physical): 512B/512B
> Partition Table: unknown
> Disk Flags:
>
> Error: /dev/nbd2: unrecognised disk label
> Model: Unknown (unknown)
> Disk /dev/nbd2: 4096B
> Sector size (logical/physical): 512B/512B
> Partition Table: unknown
> Disk Flags:
>
> Model: SD SL08G (sd/mmc)
> Disk /dev/mmcblk0: 7948MB
> Sector size (logical/physical): 512B/512B
> Partition Table: msdos
> Disk Flags:
>
> Number Start End Size Type File system Flags
> 1 4194kB 273MB 268MB primary fat32 lba
> 2 273MB 7948MB 7676MB primary ext4
>
> I’ve been trying to get them to talk for 2 days now
> One thing I did notice is that the port numbers in the config file don’t seem
> to change the fact it’s listening on he default port??
>
> $ sudo lsof -i -P -n | grep LISTEN
> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> cnid_meta 312 root 3u IPv6 11170 0t0 TCP [::1]:4700 (LISTEN)
> afpd 324 root 3u IPv4 11195 0t0 TCP *:548 (LISTEN)
> sshd 426 root 3u IPv4 12126 0t0 TCP *:22 (LISTEN)
> sshd 426 root 4u IPv6 12128 0t0 TCP *:22 (LISTEN)
> dnsmasq 431 dnsmasq 5u IPv4 12046 0t0 TCP *:53 (LISTEN)
> dnsmasq 431 dnsmasq 7u IPv6 12048 0t0 TCP *:53 (LISTEN)
> nbd-serve 2026 nbd 3u IPv6 20100 0t0 TCP *:10809 (LISTEN)
>
> I’d be very grateful for any pointers, or other resources that my be of help.
> Thanks in advance,
> Simon
>
>
>
>
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
Reply to: