[Gluster-users] One node goes offline, the other node can't see the replicated volume anymore
Greg Scott
GregScott at infrasupport.com
Tue Jul 16 14:31:21 UTC 2013
I'm the first to admit, I don't understand systemd yet. I wanted to know - how do you know what services you're starting up? What's the systemd equivalent of chkconfig --list? Well, maybe I found it - just type "systemctl" with no switches. So check out the differences between fw1 and fw2. I have no idea what this means but I think it might be relevant.
First, systemctl on fw1. Notice everything is loaded and active except a mystery service named rng.service I don't think we care about. I'll paste in all 118 lines from fw1. Now scroll down and look at the extract from fw2. So nobody is forced to wade through 200+ lines of output, I'll only paste in the relevant service that failed on fw2, named firewall\x2dscripts.mount.
This same service works on fw1, fails on fw2 with a status of "No such file or directory" - status posted below. I wonder what it does? That backslash character in a filename seems fishy. The only reference I can find anywhere is in a directory, /run/systemd/generator. I'll paste in what it looks like at the very bottom.
[root at chicago-fw1 rc.d]# systemctl
UNIT LOAD ACTIVE SUB DESCRIPTION
proc-sys-fs-binfmt_misc.automount loaded active waiting Arbitrary Executable File Formats File System Automo
sys-devices-pci0...-0000:00:1b.0-sound-card0.device loaded active plugged NM10/ICH7 Family High Definition Audio Controller
sys-devices-pci0...1-0000:02:00.0-net-enp2s0.device loaded active plugged RTL8111/8168 PCI Express Gigabit Ethernet controller
sys-devices-pci0...2-0000:03:00.0-net-enp3s0.device loaded active plugged RTL8111/8168 PCI Express Gigabit Ethernet controller
sys-devices-pci0...0-0000:05:04.0-net-enp5s4.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
sys-devices-pci0...0-0000:05:06.0-net-enp5s6.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
sys-devices-pci0...0-0000:05:07.0-net-enp5s7.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
sys-devices-pci0...:0-2:0:0:0-block-sda-sda1.device loaded active plugged SanDisk_SDSSDRC032G
sys-devices-pci0...:0-2:0:0:0-block-sda-sda2.device loaded active plugged SanDisk_SDSSDRC032G
sys-devices-pci0...:0-2:0:0:0-block-sda-sda3.device loaded active plugged SanDisk_SDSSDRC032G
sys-devices-pci0...et2:0:0-2:0:0:0-block-sda.device loaded active plugged SanDisk_SDSSDRC032G
sys-devices-platform-serial8250-tty-ttyS2.device loaded active plugged /sys/devices/platform/serial8250/tty/ttyS2
sys-devices-platform-serial8250-tty-ttyS3.device loaded active plugged /sys/devices/platform/serial8250/tty/ttyS3
sys-devices-pnp0-00:0a-tty-ttyS0.device loaded active plugged /sys/devices/pnp0/00:0a/tty/ttyS0
sys-devices-pnp0-00:0b-tty-ttyS1.device loaded active plugged /sys/devices/pnp0/00:0b/tty/ttyS1
sys-devices-virtual-block-dm\x2d0.device loaded active plugged /sys/devices/virtual/block/dm-0
sys-devices-virtual-block-dm\x2d1.device loaded active plugged /sys/devices/virtual/block/dm-1
sys-devices-virtual-block-dm\x2d2.device loaded active plugged /sys/devices/virtual/block/dm-2
sys-module-configfs.device loaded active plugged /sys/module/configfs
sys-module-fuse.device loaded active plugged /sys/module/fuse
sys-subsystem-net-devices-enp2s0.device loaded active plugged RTL8111/8168 PCI Express Gigabit Ethernet controller
sys-subsystem-net-devices-enp3s0.device loaded active plugged RTL8111/8168 PCI Express Gigabit Ethernet controller
sys-subsystem-net-devices-enp5s4.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
sys-subsystem-net-devices-enp5s6.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
sys-subsystem-net-devices-enp5s7.device loaded active plugged RTL-8110SC/8169SC Gigabit Ethernet
-.mount loaded active mounted /
boot-efi.mount loaded active mounted /boot/efi
boot.mount loaded active mounted /boot
dev-hugepages.mount loaded active mounted Huge Pages File System
dev-mqueue.mount loaded active mounted POSIX Message Queue File System
firewall\x2dscripts.mount loaded active mounted /firewall-scripts
gluster\x2dfw1.mount loaded active mounted /gluster-fw1
sys-fs-fuse-connections.mount loaded active mounted FUSE Control File System
sys-kernel-config.mount loaded active mounted Configuration File System
sys-kernel-debug.mount loaded active mounted Debug File System
tmp.mount loaded active mounted Temporary Directory
cups.path loaded active waiting CUPS Printer Service Spool
systemd-ask-password-plymouth.path loaded active waiting Forward Password Requests to Plymouth Directory Watc
systemd-ask-password-wall.path loaded active waiting Forward Password Requests to Wall Directory Watch
abrt-ccpp.service loaded active exited Install ABRT coredump hook
abrt-oops.service loaded active running ABRT kernel log watcher
abrt-xorg.service loaded active running ABRT Xorg log watcher
abrtd.service loaded active running ABRT Automated Bug Reporting Tool
alsa-state.service loaded active running Manage Sound Card State (restore and store)
atd.service loaded active running Job spooling tools
auditd.service loaded active running Security Auditing Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
chronyd.service loaded active running NTP client/server
crond.service loaded active running Command Scheduler
dbus.service loaded active running D-Bus System Message Bus
fedora-loadmodules.service loaded active exited Load legacy module configuration
fedora-readonly.service loaded active exited Configure read-only root support
getty at tty1.service loaded active running Getty on tty1
glusterd.service loaded active running GlusterFS an clustered file-system server
irqbalance.service loaded active running irqbalance daemon
lvm2-lvmetad.service loaded active running LVM2 metadata daemon
lvm2-monitor.service loaded active exited Monitoring of LVM2 mirrors, snapshots etc. using dme
mcelog.service loaded active running Machine Check Exception Logging Daemon
network.service loaded active exited LSB: Bring up/down networking
rc-local.service loaded active running /etc/rc.d/rc.local Compatibility
rngd.service loaded failed failed Hardware RNG Entropy Gatherer Daemon
rpcbind.service loaded active running RPC bind service
rsyslog.service loaded active running System Logging Service
smartd.service loaded active running Self Monitoring and Reporting Technology (SMART) Dae
sshd.service loaded active running OpenSSH server daemon
systemd-journald.service loaded active running Journal Service
systemd-logind.service loaded active running Login Service
systemd-readahead-collect.service loaded active exited Collect Read-Ahead Data
systemd-readahead-replay.service loaded active exited Replay Read-Ahead Data
systemd-remount-fs.service loaded active exited Remount Root and Kernel File Systems
systemd-sysctl.service loaded active exited Apply Kernel Variables
systemd-tmpfiles-setup.service loaded active exited Recreate Volatile Files and Directories
systemd-udev-trigger.service loaded active exited udev Coldplug all Devices
systemd-udevd.service loaded active running udev Kernel Device Manager
systemd-user-sessions.service loaded active exited Permit User Sessions
systemd-vconsole-setup.service loaded active exited Setup Virtual Console
vsftpd.service loaded active running Vsftpd ftp daemon
avahi-daemon.socket loaded active listening Avahi mDNS/DNS-SD Stack Activation Socket
cups.socket loaded active listening CUPS Printing Service Sockets
dbus.socket loaded active running D-Bus System Message Bus Socket
dm-event.socket loaded active listening Device-mapper event daemon FIFOs
iscsid.socket loaded active listening Open-iSCSI iscsid Socket
iscsiuio.socket loaded active listening Open-iSCSI iscsiuio Socket
lvm2-lvmetad.socket loaded active running LVM2 metadata daemon socket
pcscd.socket loaded active listening PC/SC Smart Card Daemon Activation Socket
rpcbind.socket loaded active listening RPCbind Server Activation Socket
syslog.socket loaded active running Syslog Socket
systemd-initctl.socket loaded active listening /dev/initctl Compatibility Named Pipe
systemd-journald.socket loaded active running Journal Socket
systemd-shutdownd.socket loaded active listening Delayed Shutdown Socket
systemd-udevd-control.socket loaded active listening udev Control Socket
systemd-udevd-kernel.socket loaded active running udev Kernel Socket
dev-dm\x2d1.swap loaded active active /dev/dm-1
basic.target loaded active active Basic System
cryptsetup.target loaded active active Encrypted Volumes
getty.target loaded active active Login Prompts
local-fs-pre.target loaded active active Local File Systems (Pre)
local-fs.target loaded active active Local File Systems
multi-user.target loaded active active Multi-User System
network-online.target loaded active active Network is Online
network.target loaded active active Network
paths.target loaded active active Paths
remote-fs.target loaded active active Remote File Systems
sockets.target loaded active active Sockets
sound.target loaded active active Sound Card
swap.target loaded active active Swap
sysinit.target loaded active active System Initialization
timers.target loaded active active Timers
systemd-readahead-done.timer loaded active elapsed Stop Read-Ahead Data Collection 10s After Completed
systemd-tmpfiles-clean.timer loaded active waiting Daily Cleanup of Temporary Directories
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
110 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
lines 94-118/118 (END)
To save a headache, now just the extract from fw2
[root at chicago-fw2 ~]# systemctl
UNIT LOAD ACTIVE SUB DESCRIPTION
.
.
.
firewall\x2dscripts.mount loaded failed failed /firewall-scripts
.
.
.
[root at chicago-fw2 rc.d]# systemctl status firewall\x2dscripts.mount -n 50
firewallx2dscripts.mount
Loaded: error (Reason: No such file or directory)
Active: inactive (dead)
And here is what that mystery service file looks like. Is my dash, "-", some kind of illegal character in fstab??
::::::::::::::
firewall\x2dscripts.mount
::::::::::::::
# Automatically generated by systemd-fstab-generator
[Unit]
SourcePath=/etc/fstab
DefaultDependencies=no
After=remote-fs-pre.target
After=network.target
After=network-online.target
Wants=network-online.target
Conflicts=umount.target
Before=umount.target
Before=remote-fs.target
[Mount]
What=localhost:/firewall-scripts
Where=/firewall-scripts
Type=glusterfs
FsckPassNo=0
Options=defaults,_netdev
More information about the Gluster-users
mailing list