posix-environ borks on phantom entry

Bug #460455 reported by Faré
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
SBCL
Fix Released
Undecided
Unassigned

Bug Description

While developing XCVB, I was _sometimes_ experiencing weird errors while running run-program. I narrowed them to this:

CL-USER> (sb-impl::posix-environ)
c-string decoding error (:external-format :UTF-8):
  the octet sequence 2 cannot be decoded.
   [Condition of type SB-INT:C-STRING-DECODING-ERROR]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [ABORT] Return to SLIME's top level.
 2: [ABORT] Exit debugger, returning to top level.

Backtrace:
  0: (SB-INT:C-STRING-DECODING-ERROR :UTF-8 2)
  1: (SB-IMPL::READ-FROM-C-STRING/UTF-8 #.(SB-SYS:INT-SAP #X10038CD8A0) CHARACTER)
  2: (SB-INT:C-STRINGS->STRING-LIST #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X7FFFFFFFDE30 :TYPE (* (C-STRING :EXTERNAL-FORMAT :DEFAULT :ELEMENT-TYPE CHARACTER))>)
  3: (SB-INT:SIMPLE-EVAL-IN-LEXENV (POSIX-ENVIRON) #<NULL-LEXENV>)

If I try to force the latin1 external format instead of utf-8, I get the following:

CL-USER> (let ((SB-ALIEN::*DEFAULT-C-STRING-EXTERNAL-FORMAT* :latin1)) (sb-impl::posix-environ))
("PWD=/home/fare/cl/xcvb" "DISPLAY=:0" "TERM=dumb" "TERMCAP=" "COLUMNS=100" "EMACS=t" "INSIDE_EMACS=23.1.50.1,comint" "WHERE=B612" "found=mutt" "NETHACKOPTIONS=boulder=$,color,DECgraphics,!sparkle,hilite_pet" "XCVB_PATH=/home/fare/cl:/home/fare/.local/share/common-lisp" "EDITFILES=" "COLORTERM=y" "tty=/dev/pts/13" "BASESHLVL=2" "LOGLVL=2" "LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=01;33:so=01;35:bd=40;33;01:cd=40;33;01:ex=01;32:mi=00:or=01;36:lc=\\e[:rc=m:ec=\\e[00m:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.tar=01;31:*.tgz=01;31:*.tbz=01;31:*.tb2=01;31:*.ttz=01;31:*.taz=01;31:*.tpz=01;31:*.arj=01;31:*.zoo=01;31:*.lzh=01;31:*.zip=01;31:*.bz=01;31:*.b2=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz=01;31:*.bz2=01;31:*.tz=01;31:*.psz=01;31:*.txz=01;31:*.jpg=01;35:*.gif=01;35:*.jpeg=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35" "BROWSER=" "BACKUPDIR=/tmp/fare/backup" "EDITOR_ENC=utf-8" "METAMAIL_PAGER=/usr/bin/less" "MM_PAGER=/usr/bin/less" "MANPAGER=/usr/bin/less -s" "EDITOR=/home/fare/bin/emacs.sh" "MAILREADER=/usr/bin/mutt" "PAGER=/usr/bin/less" "BINDIR=/home/fare/bin" "REPLYTO=Francois-Rene Rideau <email address hidden>" "SLATE_ROOT=/home/fare/src/slate/main" "NICKNAME=Fare" "IRCNAME=http://fare.tunes.org/" "WWW_HOME=file:/home/fare/etc/www/index.html" "FARE=/home/fare" "FORTUNE=/usr/games/fortune" "XLIBDIR=/usr/X11R6/lib/X11" "MAILCAP=/etc/mailcap" "INFOPATH=/usr/share/info:/usr/info:/usr/local/info" "TEXINPUTS=.:/home/fare/fare/texstuff:/usr/local/share/texmf//:" "MYFORTUNES=/usr/local/share/games/fortunes/fare" "FORTUNES=/usr/local/share/games/fortunes" "MYSRC=/usr/src" "TUNESRC=/home/fare/etc/tunes/B612" "TUNES=/usr/src/tunes" "CVS_RSH=ssh" "RSYNC_RSH=ssh -x" "TMP=/tmp/fare" "MYTMP=/tmp/fare" "DOWNLOAD_DIR=/DL" "BACKSPACE=" "LESS=-fMMQQi" "BINTYPE=x86_64" "MANPATH=/usr/local/man:/usr/share/man:/usr/man:/usr/X11R6/man" "MYETC=/home/fare/etc" "LOCALSYSTEM=B612" "ORIG_PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games" "ZDOTDIR=/home/fare" "OLDPWD=/home/fare" "WINDOWID=50331671" "KONSOLE_DBUS_SESSION=/Sessions/1" "KONSOLE_DBUS_SERVICE=:1.33" "COLORFGBG=15;0" "PROFILEHOME=" "LANGUAGE=" "DESKTOP_STARTUP_ID=" "GTK2_RC_FILES=/etc/gtk-2.0/gtkrc:/home/fare/.gtkrc-2.0::/home/fare/.kde/share/config/gtkrc-2.0" "GTK_RC_FILES=/etc/gtk/gtkrc:/home/fare/.gtkrc::/home/fare/.kde/share/config/gtkrc" "SESSION_MANAGER=local/B612:@/tmp/.ICE-unix/5558,unix/B612:/tmp/.ICE-unix/5558" "KDE_MULTIHEAD=false" "_=/usr/bin/detachtty" "XAUTHORITY=/home/fare/.Xauthority" "GTK_IM_MODULE=scim-bridge" "QT_PLUGIN_PATH=/home/fare/.kde/lib/kde4/plugins/:/usr/lib/kde4/plugins/" "WINDOWPATH=7" "LC_CTYPE=en_US.UTF-8" "XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/usr/share/gdm/" "DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-WWmeHlsB8N,guid=6fbad145130d800cad97f5aa4ae1b90d" "LOGNAME=fare" "XCURSOR_THEME=Oxygen_Black" "KDE_SESSION_VERSION=4" "SHLVL=4" "HOME=/home/fare" "GDMSESSION=kde" "GDM_LANG=en_US.UTF-8" "LANG=en_US.UTF-8" "KDE_SESSION_UID=1000" "XMODIFIERS=@im=SCIM" "GDM_XSERVER_LOCATION=local" "QT_IM_MODULE=scim-bridge" "PATH=/home/fare/bin/x86_64:/home/fare/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/local/bin/x86_64:/local/bin:/usr/local/sbin:/usr/sbin:/sbin:/conf/generic:/usr/local/games/bin:/usr/games:/usr/games/X11:/usr/games" "DESKTOP_SESSION=kde" "USERNAME=fare" "SSH_AUTH_SOCK=/tmp/ssh-xpKIZs5320/agent.5320" "USER=fare" "LC_ALL=en_US.UTF-8" "KDE_FULL_SESSION=true" "GS_LIB=/home/fare/.fonts" "SSH_AGENT_PID=5410" "SHELL=/bin/zsh" "WINDOW=0" "STY=6491.X" "Ð")

Notice the last entry, which is a string with a single character #\xD0, possibly an artefact of SBCL being confused as to where the environment ends.

/usr/bin/env, when called from emacs or the shell, does not have this invalid environment entry (invalid because it doesn't even have a = delimiter).

For reference, I get this behavior on SBCL 1.0.31.32 on Linux amd64.

Revision history for this message
Nikodemus Siivola (nikodemus) wrote :

Can you provide a test-case that sets up an environment where SBCL gets a bogus entry like this?

Revision history for this message
Faré (fahree) wrote :

If I try at the command line, I can't reproduce on demand, but if I remember correctly, was getting the error once in a while when running batch tests of XCVB before I inserted my workaround.

However, it pretty much happens to me almost all the time when running under SLIME. In a same run, I just got " ", a series of "4" then a series of "Z^K" (that's two characters, Z and Control-K) as the phantom entry, and finally a c-string decoding error (:external-format :UTF-8). So it varies during a same run.

Running 1.0.32.30 + random patch on linux/amd64 with SLIME 2009-11-06 on Emacs 23.1.50.1 from debian.

It may or may not be of note that I run in a UTF-8 environment and that SLIME somehow specifies :coding-system "iso-latin-1-unix" in its call to swank::start-server.

If you have trouble reproducing at home, I can try to get you access to a machine where I can reproduce the bug (which is even easier if you have access to systems at my current employer's).

Revision history for this message
Faré (fahree) wrote :

OK, so I disabled my workaroud and tried again until after a few attempts I got a failure again.

Backtrace:
  0: (SB-INT:C-STRING-DECODING-ERROR :UTF-8 1)
  1: (SB-IMPL::READ-FROM-C-STRING/UTF-8 #<system area pointer: #X10036C1C50> CHARACTER)
  2: (SB-INT:C-STRINGS->STRING-LIST #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X7FFFFFFFDE30 :TYPE (* (C-STRING :EXTERNAL-FORMAT :DEFAULT :ELEMENT-TYPE CHARACTER))>)

* (sb-impl::wrapped-environ)
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X7FFFFFFFDE30 :TYPE (* (C-STRING :EXTERNAL-FORMAT :DEFAULT :ELEMENT-TYPE CHARACTER))>
* (format t "~&~{ ~16,'0X~%~}~&" (loop :for i :below 102 :collect (sb-sys:sap-ref-64 (sb-sys:int-sap #X7FFFFFFFDE30) (* i 8))))
 00007FFFFFFFE2C7
 00007FFFFFFFE2D6
 00007FFFFFFFE2E1
...
 00007FFFFFFFEFA8
 00007FFFFFFFEFB7
 00007FFFFFFFEFC0
 00000010036C1C50
 0000000000000000
NIL
* (coerce (loop :for i :below 30 :collect (code-char (sb-sys:sap-ref-8 (sb-sys:int-sap #x00000010036C1C50) i))) 'string)
"'2o^C^P^@^@^@^W^@^P ^@^@^@^@ª^@^@^@^@^@^@^@`^@^@^@^@^@"

My working hypothesis is that my workaround to a previous problem caused this. In XCVB, I have:
main.lisp: #+sbcl (sb-posix:putenv (strcat "SBCL_HOME=" *lisp-implementation-directory*))
and when I query (sb-ext:posix-environ) early enough the last entry is indeed "SBCL_HOME=/usr/local/lib/sbcl/".
But at one time or another, garbage collection happens and moved the string (what else but GC would have such effect?).

If this hypothesis is correct, then the bug is with putenv using a link to a Lisp-heap-allocated string instead of using malloc and copying the string to a safe stable place before to add it to the environment.

Revision history for this message
Faré (fahree) wrote :

Note that setenv and unsetenv seem to be better interfaces to provide than putenv, and do the malloc'ing for you. It would be nice if instead of fixing putenv (that OBVIOUSLY no one is using), sb-posix would instead just remove it and add setenv / unsetenv in its place.

Revision history for this message
Faré (fahree) wrote :

Manually tested to work and survive gc on sbcl 1.0.32.30 amd64, unlike putenv.

Revision history for this message
Nikodemus Siivola (nikodemus) wrote : Re: [Bug 460455] Re: posix-environ borks on phantom entry

  status fixcommitted

Thanks! Fixed in 1.0.33.21.

Changed in sbcl:
status: New → Fix Committed
Revision history for this message
Faré (fahree) wrote :

Woohoo! Thanks a lot.

Changed in sbcl:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.