Apache for MPE/iX Troubleshooting Tips Here is some information that should be helpful when trying to troubleshoot Apache problems on MPE/iX 6.0 and 6.5. The "Apache Troubleshooting Checklist" contains a list of key items to check for problems during Apache installation and operation. For new Apache installations, your troubleshooting strategy should be to first check the network setup then the Apache setup. Part I of the checklist covers setup problems. Part II of the checklist covers areas to check once Apache is running. Apache Environment ================== The following client and server information may be useful when assessing Apache-related problems: - browser type and version (IE4, IE 5, Netscape 4.72) - browser platform (Windows 95, Windows 98, NT, HP-UX) - type of network access (dial-up, lan, via a proxy server) - MPE/iX platform (6.0, 6.5) - Apache version (/APACHE/PUB/HTTPD -v) - JHTTPD.PUB.APACHE spoolfile error messages - MPE and networking patches applied (installation verified) - amount of HPe3000 memory - content of access_log - content of error_log - content of httpd.conf - file permissions on Apache files and user files - file types (i.e BA) of Apache files and user files - DNS configuration Apache Troubleshooting Checklist ================================ I. Apache Startup problems A. Check network setup 1. Make sure the port used by Apache is not in use by another network service. Apache uses port 80 by default. This can be changed using the "Port" directive in the httpd.conf file. 2. Make sure the IP address and hostname of the server are configured correctly. Ping should run successfully, "nslookup hostname" should find the host successfully, and "telnet hostname 80" should open a telnet connection. $ /usr/sbin/ping hostname -n 5 (UNIX) :xeq nettool.net.sys "ping;ping hostname;quit" (MPE) $ telnet hostname 80 (UNIX) Trying... Connected to hostname.hp.com. Escape character is '^]'. :XEQ TELNET.ARPA.SYS (MPE) Telnet Client [A6000000] (C) Hewlett-Packard Co. 1994 telnet> open hostname 80 Trying... Connected to hostname.hp.com. Escape character is '^]'. Failed to turn on single echo: 68. 3. Make sure DNS RESLVCNF is properly configured. This is important if HostnameLookups is On since this causes a reverse DNS lookup for each each request (the client IP address is resolved to a hostname for logging in the access_log file). If not properly configured, each request will eventually complete but only after timing out. 4. Make sure the lastest Network Services Transport patch is applied (NSTxxxxx). A problem may have already been fixed so it is important to have this installed. Check with the HPRC for the latest version. B. Check Apache installation and setup 1. It is best to remove the previous installation of Apache before installing a new version due to a number of tar problems(JAGab81637). These are fixed in patch MPLX37A (6.0) and MPELX37B (6.5). Apache installation scripts for 6.0 and 6.5 use tar so MPELX37 should first be installed before updating from a previous version of Apache. Failure to install this patch first may result in an LST header error, wrong file ownership, or wrong file permissions. 2. There should be a magic file, a mime.types file, and a httpd.conf file in the conf directory. These are created by copying magic.default, mime.types.default, and httpd.conf.sample to magic, mime.types, and httpd.conf respectively. All should have group read permissions for WWW.APACHE: shellix> ls -l /APACHE/PUB/conf/magic -rw-r--r-- 1 MGR.APACHE APACHE 12441 Feb 22 19:42 magic or -rw-r----- 1 MGR.APACHE APACHE 12441 Feb 9 15:46 magic 3. The logs directory should have group read, write, and execute permissions so that the access_log, the error_log, and the httpd.pid files can be created and the Apache child processes can be forked. shellix>ls -l /APACHE/PUB drwxrwxr-x 2 MGR.APACHE APACHE 2240 Mar 26 22:37 logs or drwxrwx--- 2 MGR.APACHE APACHE 2240 Apr 13 21:30 logs 4. The log files must be writeable by WWW.APACHE -rw-rw-rw- 1 WWW.APACHE APACHE 7578 Apr 13 21:49 error_log or -rw-r----- 1 WWW.APACHE APACHE 7578 Apr 13 21:49 error_log 5. The syntax of every directive in the httpd.conf file must be correct. These can be verified by running /APACHE/PUB/HTTPD -t. For syntax definitions, consult http://www.apache.org/docs/mod/directives.html. The JHTTPD spoolfile usually gives the line number in httpd.conf where the problem is occurring. shellix>/APACHE/PUB/HTTPD -t /APACHE/PUB/conf/httpd.conf 6. Check if the Apache program file can run from the command line: shellix>/APACHE/PUB/HTTPD -f /APACHE/PUB/conf/httpd.conf or :run httpd.pub.apache;info="-f /APACHE/PUB/conf/httpd.conf" 7. All configuration files should be in bytestream format (BA). To maintain this filetype, config files can be edited with vi on the e3000 or edited on a PC using Samba. Another method is to edit the files on a PC then transfer them back to the e3000 using FTP in bytestream mode: C:\WINDOWS>ftp 3000host.hp.com ftp> quote type L 8 (also known as "tenex" or "bytestream" mode) ftp> put httpdconf /APACHE/PUB/httpd.conf In general, files can be edited by changing them to FA format using the shell's "frombyte" utility, editing with a e3000 editor, then changing them back to BA format using the shell's "tobyte" utility. This does not work for the httpd.conf file (because of line wrapping) unless the LogFormat line is also modified with continuation character. shellix>frombyte /APACHE/PUB/conf/httpd.conf /APACHE/PUB/HTTPDCF shellix>exit ... edit HTTPDCF.... LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \ \"%{User-Agent}i\"" combined ... :xeq sh.hpbin.sys -L shellix>tobyte -at /APACHE/PUB/HTTPDCF /APACHE/PUB/conf/httpd.conf 8. Check if JHTTPD is running. It should not log off. Check :SHOWPROC or shellix>ps -ef | grep -i HTTPD to see if the parent and child processes are running. 9. Check for error messages in the error_log file. For the maximum number of messages, set "LogLevel debug". If Apache startup does not proceed far enough to log into error_log, check the JHTTPD spoolfile for error messages. shellix>tail /APACHE/PUB/logs/error_log II. Apache Operation Problems A. Check the error_log file Always check error_log. For debugging, set the maximum number of messages with "LogLevel debug" in the httpd.conf file. B. Check if Apache files can be accessed 1. Check if Apache's home page can be accessed from a browser. http://hostname/index.html 2. Make sure user files are under the document root. For DocumentRoot "/APACHE/PUB/htdocs", files must reside in /APACHE/PUB/htdocs. When the following is specified from the browser, http://hostname/myfiles/file1.html Apache looks for /APACHE/PUB/htdocs/myfiles/file1.html The document root can be changed using the DocumentRoot directive. 3. Check if files can be accessed across the network. Try to display Apache's home page. "/" equates to the DocumentRoot. $telnet hostname 80 (UNIX) Trying... Connected to hostname.hp.com. Escape character is '^]'. GET / HTTP/1.0 <--- user input :xeq telnet.arpa.sys (MPE) Telnet Client [A6000000] (C) Hewlett-Packard Co. 1994 telnet> toggle crlf Will send carriage returns as telnet . telnet> open hostname 80 Trying... Connected to hostname.hp.com. Escape character is '^]'. Failed to turn on single echo: 68. GET / HTTP/1.0 <--- user input 4. Check the permissions of files being accessed. Make sure WWW.APACHE has read access to static files (group "r") and execute access for CGI (group "x"). -rw-r----- 1 MGR.APACHE APACHE 1615 Apr 10 14:57 myfile.html -rwxr-x--- 1 MGR.APACHE APACHE 757 Feb 9 13:58 test-cgi 5. Check that Apache can traverse all parent directories to get to the requested file. All parent directories must have group "x" permission. drwx--x--- 2 MGR.APACHE APACHE 704 Apr 10 14:57 test 6. Check if the problem happens with both IE and Netscape browsers and with different versions of these. C. Check that file types are in bytestream format 1. All static files (.html, .txt, etc) and all image files (.gif, .jpeg, .jpg, etc) should be in bytestream format due to the behavior of the bytestream emulator. Files that aren't in bytestream may have noticeably poor performance. To look at a file's type: :listfile /APACHE/PUB/,2 or shellix>cd /APACHE/PUB shellix>find * | xargs file The POSIX "tobyte" utility will change a file to bytestream format. D. Check for appropriate patches for kill error messages If an "Operation not permitted" error occurs when using kill, shell/iX> kill -HUP `cat /APACHE/PUB/logs/httpd.pid` kill: 6750343: Operation not permitted apply the necessary patches. To execute kill as MANAGER.SYS or an SM user, install MPELX36A on 6.0 (no patch needed for 6.5). To execute kill as MGR.APACHE, install MPELX51A on 6.0 and MPELX51B on 6.5. E. Useful configuration and operating information is available by setting up the server-info and server-status handlers in httpd.conf. "Allow from" specifies the IP addresses of users allowed access to this information: SetHandler server-info Order deny,allow Deny from all Allow from nn.nn.nnn.nn ExtendedStatus On SetHandler server-status Order deny,allow Deny from all Allow from nn.nn.nnn.nn http://hostname/server-info http://hostname/server-status F. Check the access_log Check if the request was logged in the access_log and if the number of bytes logged is as expected. If so, the request was successfully completed by Apache and any problems experienced may be MPE problems. G. Run Apache with the MPE Debugger For low-level debugging, run Apache with the MPE debugger. Apache includes a special runtime option, -X, for debugging. This option starts the Apache parent process but not the child processes and make debugging easier. :run httpd.pub.apache;info="-X -f /APACHE/PUB/conf/httpd.conf";debug Apache Resources ================ Apache Software Foundation documentation (for all vendor platforms): http://www.apache.org/docs Apache for MPE/iX Home Page on Jazz: http://jazz.external.hp.com/src/apache/index.html MPE/iX 6.5 Communicator, 6.5 Configuring and Managing MPE/iX Internet Services(Chapter 9): http://docs.hp.com