advanced-linux-programming.pdf

AdvancedLinuxProgrammingContents At a GlanceIAdvanced UNIX Programmingwith Linux1Getting Started32Writing Good GNU/LinuxSoftware173Processes454Threads615Interprocess Communication95IIMastering Linux6Devices1297The/proc File System1478Linux System Calls1679Inline Assembly Code18910Security19711A Sample GNU/Linux Application219IIIAppendixesAOther Development Tools259BLow-Level I/O281CTable of Signals301DOnline Resources303EOpen Publication License Version 1.0305FGNU General Public License30900 0430 FM 5/22/01 2:32 PM Page i00 0430 FM 5/22/01 2:32 PM Page iiAdvanced LinuxProgramming201 West 103rd Street,Indianapolis,Indiana 46290An Imprint of Pearson EducationBoston Indianapolis London Munich New YorkSan Francisco Mark Mitchell,Jeffrey Oldham,and Alex S00 0430 FM 5/22/01 2:32 PM Page iiiPublisherDavid DwyerAssociate PublisherAl ValvanoExecutive EditorStephanie WallManaging EditorGina BrownAcquisitions EditorAnn QuinnDevelopment EditorLaura LoveallProduct MarketingManagerStephanie LaytonPublicity ManagerSusan PetroProject EditorCaroline WiseCopy EditorKrista HansingSenior IndexerCheryl LenserManufacturingCoordinatorJim ConwayBook DesignerLouisa KlucznikCover DesignerBrainstorm Design,Inc.Cover ProductionAren HowellProofreaderDebra NeelCompositionAmy ParkerAdvanced Linux ProgrammingCopyright 2001 by New Riders PublishingFIRST EDITION:June,2001All rights reserved.No part of this book may be reproducedor transmitted in any form or by any means,electronic ormechanical,including photocopying,recording,or by anyinformation storage and retrieval system,without writtenpermission from the publisher,except for the inclusion ofbrief quotations in a review.International Standard Book Number:0-7357-1043-0Library of Congress Catalog Card Number:00-10534305 04 03 02 01 7 6 5 4 3 2 1Interpretation of the printing code:The rightmost double-digit number is the year of the books printing;the right-most single-digit number is the number of the booksprinting.For example,the printing code 01-1 shows that thefirst printing of the book occurred in 2001.Composed in Bembo and MCPdigital by New RidersPublishing.Printed in the United States of America.TrademarksAll terms mentioned in this book that are known to betrademarks or service marks have been appropriately capital-ized.New Riders Publishing cannot attest to the accuracy ofthis information.Use of a term in this book should not beregarded as affecting the validity of any trademark or servicemark.PostScript is a trademark of Adobe Systems,Inc.Linux is a trademark of Linus Torvalds.Warning and DisclaimerThis book is designed to provide information aboutAdvanced Linux Programming.Every effort has been made tomake this book as complete and as accurate as possible,butno warranty or fitness is implied.The information is provided on an as-is basis.The authorsand New Riders Publishing shall have neither liability norresponsibility to any person or entity with respect to any lossor damages arising from the information contained in thisbook or from the use of the discs or programs that mayaccompany it.00 0430 FM 5/22/01 2:32 PM Page iv00 0430 FM 5/22/01 2:32 PM Page vTable of ContentsI Advanced UNIX Programmingwith Linux11Getting Started31.1Editing with Emacs41.2Compiling with GCC61.3Automating the Process with GNU Make91.4Debugging with GNU Debugger(GDB)111.5Finding More Information132Writing Good GNU/Linux Software172.1Interaction With the ExecutionEnvironment172.2Coding Defensively302.3Writing and Using Libraries363Processes453.1Looking at Processes453.2Creating Processes483.3Signals523.4Process Termination554Threads614.1Thread Creation624.2Thread Cancellation694.3Thread-Specific Data724.4Synchronization and Critical Sections774.5GNU/Linux Thread Implementation924.6Processes Vs.Threads9400 0430 FM 5/22/01 2:32 PM Page vi5Interprocess Communication955.1Shared Memory965.2Processes Semaphores1015.3Mapped Memory1055.4Pipes1105.5Sockets116II Mastering Linux1276Devices1296.1Device Types1306.2Device Numbers1306.3Device Entries1316.4Hardware Devices1336.5Special Devices1366.6PTYs1426.7ioctl1447The/proc File System1477.1Extracting Information from/proc1487.2Process Entries1507.3Hardware Information1587.4Kernel Information1607.5Drives,Mounts,and File Systems1617.6System Statistics1658Linux System Calls1678.1Using strace1688.2access:Testing File Permissions1698.3fcntl:Locks and Other File Operations1718.4fsync and fdatasync:Flushing Disk Buffers1738.5getrlimit and setrlimit:Resource Limits1748.6getrusage:Process Statistics1758.7gettimeofday:Wall-Clock Time176viiContents00 0430 FM 5/22/01 2:32 PM Page vii8.8The mlock Family:Locking PhysicalMemory1778.9mprotect:Setting Memory Permissions1798.10nanosleep:High-Precision Sleeping1818.11readlink:Reading Symbolic Links1828.12sendfile:Fast Data Transfers1838.13setitimer:Setting Interval Timers1858.14sysinfo:Obtaining System Statistics1868.15uname1879Inline Assembly Code1899.1When to Use Assembly Code1909.2Simple Inline Assembly1919.3Extended Assembly Syntax1929.4Example1949.5Optimization Issues1969.6Maintenance and Portability Issues19610Security19710.1Users and Groups19810.2Process User IDs and Process Group IDs19910.3File System Permissions20010.4Real and Effective IDs20510.5Authenticating Users20810.6More Security Holes21111A Sample GNU/Linux Application21911.1Overview21911.2Implementation22111.3Modules23911.4Using the Server25211.5Finishing Up255viiiContents00 0430 FM 5/22/01 2:32 PM Page viiiIII Appendixes257AOther Development Tools259A.1Static Program Analysis259A.2Finding Dynamic Memory Errors261A.3Profiling269BLow-Level I/O281B.1Reading and Writing Data282B.2stat291B.3Vector Reads and Writes293B.4Relation to Standard C Library I/OFunctions295B.5Other File Operations296B.6Reading Directory Contents296CTable of Signals301DOnline Resources303D.1General Information303D.2Information About GNU/Linux Software304D.3Other Sites304EOpen Publication License Version 1.0305I.Requirement on Both Unmodified andModified Versions 305II.Copyright306III.Scope of License306IV.Requirements on Modified Works 306V.Good-Practice Recommendations 306VI.License Options 307Open Publication Policy Appendix 307ixContents00 0430 FM 5/22/01 3:18 PM Page ixFGNU General Public License309Preamble309Terms and Conditions for Copying,Distribution and Modification310End of Terms and Conditions315How to Apply These Terms to Your NewPrograms315Index317xContents00 0430 FM 5/22/01 2:32 PM Page x00 0430 FM 5/22/01 2:32 PM Page xiTable of Program Listings1.1 main.c(C source file),61.2 reciprocal.cpp(C+source file),61.3 reciprocal.hpp(header file),72.1 arglist.c(argc and argv parameters),182.2 getopt_long.c(getopt_long function),212.3 print_env.c(printing execution environment),262.4 client.c(network client program),262.5 temp_file.c(mkstemp function),282.6 readfile.c(resource allocation during errorchecking),352.7 test.c(library contents),372.8 app.c(program with library functions),372.9 tifftest.c(libtiff library),403.1 print-pid.c(printing process IDs),463.2 system.c(system function),483.3 fork.c(fork function),493.4 fork-exec.c(fork and exec functions),513.5 sigusr1.c(signal handlers),543.6 zombie.c(zombie processes),583.7 sigchld.c(cleaning up child processes),604.1 thread-create.c(creating threads),634.2 thread-create2(creating two threads),644.3 thread-create2.c(revised main function),654.4 primes.c(prime number computation in athread),674.5 detached.c(creating detached threads),694.6 critical-section.c(critical sections),714.7 tsd.c(thread-specific data),734.8 cleanup.c(cleanup handlers),754.9 cxx-exit.cpp(C+thread cleanup),764.10 job-queue1.c(thread race conditions),784.11 job-queue2.c(mutexes),804.12 job-queue3.c(semaphores),844.13 spin-condvar.c(condition variables),8700 0430 FM 5/22/01 2:32 PM Page xii4.14 condvar.c(condition variables),904.15 thread-pid(printing thread process IDs),925.1 shm.c(shared memory),995.2 sem_all_deall.c(semaphore allocation anddeallocation),1025.3 sem_init.c(semaphore initialization),1025.4 sem_pv.c(semaphore wait and post operations),1045.5 mmap-write.c(mapped memory),1065.6 mmap-read.c(mapped memory),1075.7 pipe.c(parent-child process communication),1115.8 dup2.c(output redirection),1135.9 popen.c(popen command),1145.10 socket-server.c(local sockets),1205.11 socket-client.c(local sockets),1215.12 socket-inet.c(Internet-domain sockets),1246.1 random_number.c(random number generation),1386.2 cdrom-eject.c(ioctl example),1447.1 clock-speed.c(cpu clock speed from/proc/cpuinfo),1497.2 get-pid.c(process ID from/proc/self),1517.3 print-arg-list.c(printing process argumentlists),1537.4 print-environment.c(process environment),1547.5 get-exe-path.c(program executable path),1557.6 open-and-spin.c(opening files),1577.7 print-uptime.c(system uptime and idle time),1658.1 check-access.c(file access permissions),1708.2 lock-file.c(write locks),1718.3 write_journal_entry.c(data buffer flushing),1738.4 limit-cpu.c(resource limits),1758.5 print-cpu-times.c(process statistics),176xiiiProgram Listings00 0430 FM 5/22/01 2:32 PM Page xiii8.6 print-time.c(date/time printing),1778.7 mprotect.c(memory access),1808.8 better_sleep.c(high-precision sleep),1828.9 print-symlink.c(symbolic links),1838.10 copy.c(sendfile system call),1848.11 itemer.c(interal timers),1858.12 sysinfo.c(system statistics),1878.13 print-uname(version number and hardware information),1889.1 bit-pos-loop.c(bit position with loop),1949.2 bit-pos-asm.c(bit position with bsrl),19510.1 simpleid.c(printing user and group IDs),20010.2 stat-perm.c(viewing file permissions withstat system call),20210.3 setuid-test.c(setuid programs),20710.4 pam.c(PAM example),20910.5 temp-file.c(temporary file creation),21410.6 grep-dictionary.c(word search),21611.1 server.h(function and variable declarations),22211.2 common.c(utility functions),22311.3 module.c(loading server modules),22611.4 server.c(server implementation),22811.5 main.c(main server program),23511.6 time.c(show wall-clock time),23911.7 issue.c(GNU/Linux distribution information),24011.8 diskfree.c(free disk space information),24211.9 processes.c(summarizing runningprocesses),24411.10 Makefile(Makefile for sample applicationprogram),252xivProgram Listings00 0430 FM 5/22/01 2:32 PM Page xivA.1 hello.c(Hello World),260A.2 malloc-use.c(dynamic memory allocation),267A.3 calculator.c(main calculator program),274A.4 number.c(unary number implementation),276A.5 stack.c(unary number stack),279A.6 definitions.h(header file for calculator program),280B.1 create-file.c(create a new file),284B.2 timestamp.c(append a timestamp),285B.3 write-all.c(write all buffered data),286B.4 hexdump.c(print a hexadecimal file dump),287B.5 lseek-huge.c(creating large files),289B.6 read-file.c(reading files into buffers),292B.7 write-args.c(writev function),294B.8 listdir.c(printing directory listings),297xvProgram Listings00 0430 FM 5/22/01 2:32 PM Page xvxviAbout the AuthorsMark Mitchell received a bachelor of arts degree in computer science from Harvard in 1994 and a master of science degree fromStanford in 1999.His research interests centered on computationalcomplexity and computer security.Mark has participated substantiallyin the development of the GNU Compiler Collection,and he has astrong interest in developing quality software.Jeffrey Oldham received a bachelor of arts degree in computer science from Rice University in 1991.After working at the Center forResearch on Parallel Computation,he obtained a doctor of philoso-phy degree from Stanford in 2000.His research interests center onalgorithm engineering,concentrating on flow and other combinator-ial algorithms.He works on GCC and scientific computing software.Alex Samuel graduated from Harvard in 1995 with a degree inphysics.He worked as a software engineer at BBN before returningto study physics at Caltech and the Stanford Linear AcceleratorCenter.Alex administers the Software Carpentry project and workson various other projects,such as optimizations in GCC.Mark and Alex founded CodeSourcery LLC together in 1999.Jeffrey joined the company in 2000.CodeSourcerys mission is toprovide development tools for GNU/Linux and other operating systems;to make the GNU tool chain a commercial-quality,standards-conforming development tool set;and to provide generalconsulting and engineering services.CodeSourcerys Web site ishttp:/.00 0430 FM 5/22/01 2:32 PM Page xvixviiAbout the Technical ReviewersThese reviewers contributed their considerable hands-on expertise to the entire devel-opment process for Advanced Linux Programming.As the book was being written,thesededicated professionals reviewed all the material for technical content,organization,and flow.Their feedback was critical to ensuring that Advanced Linux Programming fitsour readers need for the highest quality technical information.Glenn Becker has many degrees,all in theatre.He presently works as anonline producer for SCIFI.COM,the online component of the SCI FIchannel,in New York City.At home he runs Debian GNU/Linux andobsesses about such topics as system administration,security,softwareinternationalization,and XML.John Dean received a BSc(Hons)from the University of Sheffield in1974,in pure science.As an undergraduate at Sheffield,John developedhis interest in computing.In 1986 he received a MSc from CranfieldInstitute of Science and Technology in Control Engineering.While work-ing for Roll Royce and Associates,John became involved in developingcontrol software for computer-aided inspection equipment of nuclearsteam-raising plants.Since leaving RR&A in 1978,he has worked in thepetrochemical industry developing and maintaining process control soft-ware.John worked a volunteer software developer for MySQL from 1996until May 2000,when he joined MySQL as a full-time employee.Johns area ofresponsibility is MySQL on MS Windows and developing a new MySQL GUI client using Trolltechs Qt GUI application toolkit on both Windows and platforms that run X-11.00 0430 FM 5/22/01 2:32 PM Page xviixviiiAcknowledgmentsWe greatly appreciate the pioneering work of Richard Stallman,without whom there would never have been the GNU Project,and of Linus Torvalds,without whom there would never have been the Linux kernel.Countless others have workedon parts of the GNU/Linux operating system,and we thank them all.We thank the faculties of Harvard and Rice for our undergraduate educations,andCaltech and Stanford for our graduate training.Without all who taught us,we wouldnever have dared to teach others!W.Richard Stevens wrote three excellent books on UNIX programming,and we haveconsulted them extensively.Roland McGrath,Ulrich Drepper,and many others wrotethe GNU C library and its outstanding documentation.Robert Brazile and Sam Kendall reviewed early outlines of this book and made won-derful suggestions about tone and content.Our technical editors and reviewers(espe-cially Glenn Becker and John Dean)pointed out errors,made suggestions,and providedcontinuous encouragement.Of course,any errors that remain are no fault o