Menu
September 12, 2016 1. Check Point Administrator Study Guide. CCSA OBJECTIVES. Check Point technology is designed to address network exploitation, administrative flexibil ity and critical accessibility. This Section introduces the basic concepts of network security and management based on Check Point’s three- tier structure.
- Checkpoint 1 1 – Verify Without Copying Permission Document
- Checkpoint 1 1 – Verify Without Copying Permission Form
- Checkpoint 1 1 – Verify Without Copying Permission File
- Checkpoint 1 1 – Verify Without Copying Permission Documents
- The Certificate Authority Certificate View window displays the SHA-1 Fingerprint (hash) of the Internal CA certificate. Example: Copy the SHA-1 Fingerprint (e.g., to a Notepad). On Security Gateway / each cluster member: Important Note: In cluster environment, this procedure must be performed on all members of the cluster. Connect to command line.
- Upgrade the Check Point Management Server to higher version by installing upgrade package. Export configuration database using migrate utility, clean install higher version of Check Point, import the configuration that was exported earlier.
The SunOS command line is used to manipulate files and directories. You type in the file and directory names in conjunction with SunOS commandsto carry out specific operations. This is different than using the OpenWindowsFile Manager, where files are displayed as icons that can be clicked on andmoved, and commands are selected from menus.
This chapter introduces you to the concepts and procedures used to workwith files and directories from the SunOS command line. These operations applyto any SunOS command line, whether you are using a Shell or Command Tool inthe OpenWindows environment or are logged in from a remote terminal. To fullymake use of the SunOS operating system it is essential for you to understandthe concepts presented in this chapter.
The file is the basic unit in the SunOS operatingsystem. Almost everything is treated as a file, including:
- Documents--These include textfiles, such as letters or reports, computer source code, or anything elsethat you write and want to save.
- Commands--Most commandsare executable files; that is, they are files you canexecute to run a particular program. For example, the datecommand that you saw in the previous chapter, which executes a program thatprovides the current date, is an executable file.
- Devices--Your terminal,printer, and disk drive(s) are all treated as files.
- Directories--A directoryis simply a file that contains other files.
The following section explains the commands available for creating,listing, copying, moving, and deleting files. You'll also see how to listthe contents of a file and how to determine the nature of a file.
Each of the commands presented in this section includes an example ofhow the command is used. Try the examples as you read the text. This practicewill make the commands and their respective concepts easier to understandand remember.
3.2.1 Before You Begin
Before you start experimenting with files, make sure that you are inyour home directory. This is a directory establishedfor you by your system administrator when your account was created. If youperform the tasks shown in the following examples from your home directory,you'll be less likely to create, copy, move, or (worst of all) delete fileswithin portions of the system that other users expect to remain unchanged.
To make certain that you are indeed in your home directory, type the cd (change directory) command by itself. This moves you to yourhome (default) directory. Then type the pwd (print workingdirectory) command to display your current location within the filesystem.The directory displayed is your home directory:
In this example, the user's home directory is /export/home/username, where username is the name of the user owning the home directory.
3.2.2 Creating a Test File
Use the touch command to create an empty file. Ifa file by the name you specify doesn't already exist, the touchcommand creates an empty file (if the file already exists, touch updates the last file access time).
3.2.3 Listing Files (ls)
Now list the file with the ls command to verify thatyou've created it:
When you enter the ls command by itself, it listsall the files in your current location. If you enter the lscommand with a specific file name, it lists only that file, if the file exists.
For more information on the ls(1) command, refer to the man Pages(1):User Commands.
3.2.4 Copying Files (cp)
Use the cp command to copy tempfileto a file called copyfile:
Now try listing both files. Notice that both names end with the characters 'file.'You can use the wildcard character, asterisk (*), tostand for any character or sequence of characters. Therefore, the command ls*file should list both tempfile and copyfile (and any other file in thisdirectory with a name that ends with file):
Notice that copyfile is listed first. Files arelisted in alphabetical order. (Capital letters and numbers precede lowercaseletters.)
For detailed information on the cp(1) command, refer to the man Pages(1): User Commands.
3.2.5 Moving and Renaming Files (mv)
You can both move and rename files using the same command, mv (move). In this example, use the mv commandto rename tempfile to emptyfile:
Now list both files again to verify the change:
As you can see, tempfile is replaced by emptyfile.
For more information on the mv(1) command, refer to the man Pages(1):User Commands.
3.2.6 Deleting Files (rm)
Finally, use the rm (remove) command to delete copyfile, and verify the result with the lscommand:
Caution - Once you delete a file, it is gone for good. Unless there is a backupcopy, the file cannot be restored. Be careful when using the rmcommand, and be particularly careful when using rm withthe wildcard character (*). Files removed with rm cannot be recovered.
For more detailed information on the rm(1) command, refer to the man Pages(1):User Commands.
3.2.7 Displaying File Contents (more, cat)
Use the more command to display the contents of afile. Type more followed by the name of the file to bedisplayed. The contents of the file scrolls down the screen. If the file islonger than one screen, this message appears:
where nn is the percentage of the file alreadydisplayed.
You can also use the cat command to display the contentsof a file, but it flashes through the entire file rapidly without pausing.The cat (concatenate) command is more often used to jointwo or more files into one large file, as in this example:
For further information on the more(1) or cat(1) commands, refer to the manPages(1): User Commands.
3.2.8 Displaying File Type (file)
Some files, such as binary files and executable files, are not printableand cannot be displayed on the screen. The file commandcan be handy if you're not sure of the file type.
Use the file command to show the file type:
By now you know how to list, copy, rename, and delete files. However,you may be wondering about larger issues. Where are these files located? Thissection discusses the directory hierarchy. Read the following narrative carefully,and then try the examples in the sections that follow.
3.3.1 Directory Hierarchy
Files are grouped into directories, which are themselves organized ina hierarchy. At the top of the hierarchy is the 'root' directory,symbolized by '/'.
As shown in the following example, Figure 3-1, each directory in the file system can have many directories withinit. The convention is to distinguish directory levels with the / character. With this in mind, notice that the directory / (root) contains the subdirectories /usr, /bin, /home and /lib,among others. The subdirectory /home contains user1, user2, and user3.
You specify directories (and files within them) by including the namesof the directories they're in. This is called a path name.For example, the path name for the user3 directory inthe illustration above is /home/user3.
Figure 3-1 File System Hierarchy
All subdirectory and file names within a directory must be unique. However,names within different directories can be the same. For example, the directory /usr contains the subdirectory /usr/lib. Thereis no conflict between /usr/lib and /libbecause the path names are different.
Path names for files work exactly like path names for directories. Thepath name of a file describes that file's place within the file system hierarchy.For example, if the /home/user2 directory contains afile called report5, the path name for this file is /home/user2/report5. This shows that the file report5 is within the directory user2, which is withinthe directory home, which is within the root (/) directory.
Directories can contain only subdirectories, only files, or both.
3.3.2 Print Working Directory (pwd)
The command pwd (print working directory) tells youwhere you are in the file system hierarchy:
Your output will look somewhat different from that in the example, asyour directory structure will be different. Remember that your working directoryis your current location within the file system hierarchy.
3.3.3 Your Home Directory
Every user has a home directory. When you firstopen the Command Tool or Shell Tool window in the OpenWindows environment,your initial location (working directory) is your home directory. This directoryis established for you by the system administrator when your account is created.
3.3.4 Change Working Directory (cd)
The cd (change directory) command allows you to movearound within the file system hierarchy:
When you type the cd command by itself, you returnto your home directory. For example, if your home directory was /home/user1:
In the C shell, the tilde (~) is used as a shortcutfor specifying your home directory. For example, you would type the followingto change to the subdirectory music within your homedirectory:
![Copying Copying](https://sc1.checkpoint.com/sc/SolutionsStatics/sk104418/Set_Role_Name.png)
You can also use this shortcut to specify another user's home directory.For example:
where username is another user's login name, wouldchange to that user's home directory.
Note - If you are using the Bourne shell, the ~ shortcut will not work.
If you are using the Bourne shell, it may be possible that your systemadministrator has configured the system so that you can type
$home
to specify your home directory. If this is the case, thentyping: changes you to the subdirectory music in your homedirectory. Likewise, typing:
changes you to the specified user's home directory, where username represents another user's login name.
The directory immediately 'above' a subdirectory is calledthe parent directory. In the preceding example, /home is the parent directory of /home/user1.The symbol . ('dot-dot') represents theparent directory. Therefore, the command cd . changesthe working directory to the parent directory, as in this example:
Suppose your current working directory is /home/user1and you want to work with some files in /home/user2.Here is a useful shortcut:
./user2 tells the system to look in the parentdirectory for user2. As you can see, this is much easierthan typing the entire path name /home/user2.
3.3.5 Creating a Directory (mkdir)
It is easy to create a new directory. Type the mkdircommand followed by the name of the new directory:
3.3.6 Relative Path Names
The full path name of a directory or a file begins with a slash (/) and describes the entire directory structure between that file(or directory) and the root directory. However, you can often use a much shortername which defines the file or directory relative tothe current working directory.
When you are in a parent directory, you can move to a subdirectory usingonly the directory name and not the full path name. In the previous example,the command cdveggies uses the relativepath name of the directory veggies. If the current workingdirectory is /home/user2, the full path name of thisdirectory is /home/user2/veggies.
Try creating several different subdirectories, and then move aroundwithin this directory structure. Use both full path names and relative pathnames, and confirm your location with the pwd command.
3.3.7 Moving and Renaming Directories
You rename a directory by moving it to a different name. Use the mv command to rename directories:
You can also use mv to move a directory to a locationwithin another directory:
In this example, the directory carrots is movedfrom veggies to veggies2 with the mv command.
3.3.8 Copying Directories
Use the cp-r command to copy directoriesand the files they contain:
This command copies all files and subdirectories within the directory veggies to a new directory veggies3. Thisis a recursive copy, as designated by the -r option. If you attempt to copy a directory without using this option,you will see an error message.
3.3.9 Removing Directories (rmdir)
To remove an empty directory, use the rmdir commandas follows:
If the directory still contains files or subdirectories, the rmdir command will not remove the directory.
Use rm-r (adding the recursive option -r to the rmcommand) to remove a directory and all its contents, including any subdirectoriesand their files, as follows:
Caution - Directories removed with the rmdir command cannot be recovered, nor can directories andtheir contents removed with the rm-rcommand.
It often happens that different people with access to a file make copiesof the file and then edit their copies. diff will showyou the specific differences between versions of an ASCII file. The command:
scans each line in leftfile and rightfile looking for differences. When it finds a line (or lines) thatdiffer, it determines whether the difference is the result of an addition,a deletion, or a change to the line, and how many lines are affected. It tellsyou the respective line number(s) in each file, followed by the relevant textfrom each.
If the difference is the result of an addition, diffdisplays a line of the form:
where l is a line number in leftfile and r is a line number in rightfile.
If the difference is the result of a deletion, diffuses a d in place of a; if it is theresult of a change on the line, diff uses a c. Corel mydvd pro 1 2 8 download.
The relevant lines from both files immediately follow the line numberinformation. Text from leftfile is preceded by a leftangle bracket (<). Text from rightfileis preceded by a right angle bracket (>).
This example shows two sample files, followed by their diff output:
If the two files to be compared are identical, there is no output from diff.
Memorytamer 1 5 1 – automatic memory freeing apple. The diff(1)commandhas many more options than those discussed here. For more information, referto the man Pages(1): User Commands.
3.4.1 Comparing Three Different Files (diff3)
If you have three versions of a file that you want to compare at once,use the diff3 command as follows:
diff3 compares three versions of a file and publishesdisagreeing ranges of text flagged with these codes:
all three files differ
1 file1 is different
2 file2 is different
3 file3 is different
3.4.2 Using bdiff on Large Files
If you are comparing very large files, use bdiffinstead of diff. Both programs work in a similar manner:
Use bdiff instead of diff forfiles longer than 3500 lines or so.
The find command searches for files that meet conditionsyou specify, starting from a directory you name. For example, you might searchfor filenames that match a certain pattern or that have been modified withina specified time frame.
Unlike most commands, find options are several characterslong, and the name of the starting directory must precede them on the commandline as follows:
where directory is the name of the startingdirectory and options represents the options forthe find command.
Each option describes a criterion for selecting a file. A file mustmeet all criteria to be selected. Thus, the more options you apply, the narrowerthe field becomes. The -print option indicates that you wantthe results to be displayed. (As described later on, you can use find to run commands. You may want find to omitthe display of selected files in that case.)
The -namefilename optiontells find to select files that match filename. Here filename is taken to be therightmost component of a file's full path name. For example, the rightmostcomponent of the file /usr/lib/calendar is calendar. This portion of a file's name is often called its base name.
For example, to see which files within the current directory and itssubdirectories end in s, type:
Other options include:
-namefilename
Selects files whose rightmost component matches filename. Surround filename with single quotesif it includes filename substitution patterns.
-useruserid
Selects files owned by userid. userid can be either a login name or user ID number.
-groupgroup
Selects files belonging to group.
-m-timen
Selects files that have been modified within ndays.
-newercheckfile
Selects files modified more recently than checkfile.
You can specify an order of precedence by combining options within (escaped)parentheses (for example, (options) ). Within escapedparentheses, you can use the -o flag between options to indicatethat find should select files that qualify under eithercategory, rather than just those files that qualify under both categories:
You can invert the sense of an option by prepending an escaped exclamationpoint. find then selects files for which the option does not apply:
You can also use find to apply commands to the filesit selects with the
-execcommand'{}' ;
option. This option is terminated with an escaped semicolon (;). The quoted braces are replaced with the filenames that find selects.
As an example, you can use find to automaticallyremove temporary work files. If you name your temporary files consistently,you can use find to seek them out and destroy them wherever they lurk. Forexample, if you name your temporary files junk or dummy, this command will find them and remove them:
For more information on find(1), refer to the man Pages(1):User Commands.
Note - Read this section carefully. A clear understanding of file permissionsis often important in day-to-day work.
File permissions help to protect files and directories from unauthorizedreading and writing. Often you will have files you wish to allow others toread but not change. In other cases, you may have executable files (programs)to share. File permissions allow you to control access to your files.
These are the basic file and directory permission types:
- r - read permission. A file mustbe readable to be examined or copied. A directory must be readable for youto list its contents.
- w - write permission.A file must be writable in order for you to modify it, remove it, or renameit. A directory must be writable in order for you to add or delete files init.
- x - execute permission.A file with executable permissions is one you can run, such as a program.A directory must be executable for you to gain access to any of its subdirectories.
There are three categories of users for which you can set permissions:
![Checkpoint 1 1 – Verify Without Copying Permission Checkpoint 1 1 – Verify Without Copying Permission](https://dreezman.files.wordpress.com/2012/05/ldap3.jpg?w=600)
- Self - The user
- Group - Other users within the same groupas the user (for example, all accounting users). Groups are established andmaintained by the system administrator.
- Others - Everyone else
3.6.1 Displaying Permissions and Status (su ls-l)
You have already used the ls command to list files.The ls command has many options. Use the -loption to display a long format list. Files and directoriesare listed in alphabetical order. Figure 3-2 illustrates this method for displayingfiles:
Figure 3-2 Displaying Permissions and Status
Checkpoint 1 1 – Verify Without Copying Permission Document
The very first character on the line indicates the file type. A dash(-) is an ordinary file; a dindicates a directory, and other characters can indicate other special filetypes.
The next nine characters indicate the permissions for the file or directory.The nine characters consist of three groups of three, showing the permissionsfor the owner, the owner's group, and the world, respectively. The permissionsfor emptyfile are rw-r--r--,indicating that the owner can read and write this file, everyone can readit, and no one can execute it. The permissions for the directory veggies2 are rwxr-xr-x, indicatingthat everyone has read and execute permissions, but only the owner can writeto it.
Checkpoint 1 1 – Verify Without Copying Permission Form
In addition to file permissions, the display shows the following information:
- Number of links to this file or directory
- Name of the owner (user2 inthis case)
- Number of bytes (characters) in the file
- Date and time the file or directory was last updated
- Name of the file or directoryUse the cd command to move to your home directory,and try the ls-lcommand. Your resultswill differ from the example, of course.Now try typing a command such as the following:
where dirname is the name of an actual directorywithin your file system. When you give the name of a directory, the ls-l command prints information on all the filesand directories (if any) within that directory.
3.6.2 Listing 'Hidden' Files (ls-a)
There are some files that are not listed by the ordinary ls command. These files have names beginning with the character . (called 'dot'), such as .cshrc, .login and .profile. Use the ls-a command to list these dot files:
Notice that the files beginning with . are listedbefore the other files. There are two special files in this listing: the file .is the reference for the current directory, and the file . is the reference for the parent directory.
Generally speaking, files that begin with . are usedby system utilities and are not usually modified by the user. There are afew exceptions to this.
3.6.3 Changing Permissions (chmod)
Use the chmod command to change permissions for afile or directory. You must be the owner of a file or directory, or have rootaccess, to change its permissions. The general form of the chmod command is:
where permissions indicates the permissionsto be changed and name is the name of the affected file or directory.
The permissions can be specified in several ways. Here is one of theforms which is easiest to use:
- Use one or more letters indicating the users involved:
- u (for the user)
- g (for group)
- o (for others)
- a (for allthree of the above categories)
- Indicate whether the permissions are to be added(+) or removed (-).
- Use one or more letters indicating the permissionsinvolved:
- r (for read)
- w (for write)
- x (for execute)
In the following example, write permission is added to the directory carrots for users belonging to the same group (thus, permissions is g+w and nameis carrots):
As you can see, the hyphen (-) in the set of charactersfor group is changed to a w as a result of this command.
To make this same directory unreadable and unexecutable by other usersoutside your group (permissions is o-rx),you would enter the following:
Now, the r (for read) and the x(for execute) in the set of characters for other users are both changed tohyphens (-).
When you create a new file or directory, the system automatically assignspermissions.
In general, the default settings for new files are:
-rw-r--r--
and for new directories are:
drwxr-xr-x
So, to make a new file turnip executable by itsowner (user2), you would enter the following:
If you want to affect all three categories of users at once, use the -a option. To make a new file garlic executableby everyone, you would enter the following:
As a result, the x indicator appears in all threecategories.
You can also change permissions for groups of files and directoriesusing the * wildcard character. For example, you wouldenter the following to change the permissions for all the files in the currentdirectory veggies so that the files can be written byyou alone:
The pwd command is included in this example to illustratethat the directory on which you perform this chmod operationmust be the current directory.
3.6.4 Setting Absolute Permissions
Up to this point, the discussion on permissions has only included usingthe chmod command to change permissions relative to their current settings. Using a different form of the chmod command, which applies numeric codes to specify permissions,you can set the permissions for a file or directory absolutely.
The syntax for this usage of the chmod command is:
chmodnumcode name
where numcode is the numeric code and name is the name of the file or directory for which you arechanging permissions.
The complete numeric code consists of three numbers. One number is usedfor each of the three categories: user, group, and others. For example thefollowing command sets absolute read, write, and execute permissions for theuser and the group, and execute permissions only for others:
Table 3-1 illustrates how the permissions described for garlic are represented by the code 771.
Table 3-1 Permissions for garlicPermission | User | Group | Others |
---|---|---|---|
Read | 4 | 4 | 0 |
Write | 2 | 2 | 0 |
Execute | 1 | 1 | 1 |
Total | 7 | 7 | 1 |
Each of the columns in Table 3-1 represents one of the categories: user,group, and others. To set read permissions, you add 4 to the appropriate column.To set write permissions, you add 2. To add execute permissions, you add 1.The total in all three columns in the last row of the table is the completenumeric code.
The following is another example of this method for setting absolutepermissions, with the ls-l command includedto demonstrate the results:
The permissions for the file onion are set so thatthe user can read, write, and execute; group members can read and execute;and others can also read and execute. Table 3-2 provides the breakdown ofthe numeric code used to set the permissions for onion.
Table 3-2 Permissions for onionPermission | User | Group | Others |
---|---|---|---|
Read | 4 | 4 | 4 |
Write | 2 | 0 | 0 |
Execute | 1 | 1 | 1 |
Total | 7 | 5 | 5 |
Of course, to provide read, write, and execute permissions for the file cabbage to yourself, your group, and all other users, you wouldenter the following:
Table 3-3 provides the breakdown for this example.
Table 3-3 Permissions for cabbagePermission | User | Group | Others |
---|---|---|---|
Read | 4 | 4 | 4 |
Write | 2 | 2 | 2 |
Execute | 1 | 1 | 1 |
Total | 7 | 7 | 7 |
The numeric code 777 represents the maximum levelof permissions you can provide.
Similar to changing relative permissions, you can also use the wildcardcharacter * to set absolute permissions for all in thefiles in the current directory. For example, to set absolute permissions forall files in the current directory veggies so that youhave read, write, and execute permissions; your group has read and executepermissions; and all other users have execute permissions only, you wouldenter the following:
The pwd command is included in this example to illustratethat the directory on which you perform this operation must be the currentdirectory. The ls-l command is shown onlyto illustrate the changes in permissions. When setting absolute permissions,it's not necessary to know what the permissions are currently.
For more information on the chmod(1) command, refer to the manPages(1): User Commands.
Solving pg_xlog out of disk space problem on Postgres
ByGreg Sabino Mullane
September 25, 2014
September 25, 2014
Running out of disk space in the pg_xlog directory is a fairly common Postgres problem. This important directory holds the WAL (Write Ahead Log) files. (WAL files contain a record of all changes made to the database—see the link for more details). Because of the near write‑only nature of this directory, it is often put on a separate disk. Fixing the out of space error is fairly easy: I will discuss a few remedies below.
When the pg_xlog directory fills up and new files cannot be written to it, Postgres will stop running, try to automatically restart, fail to do so, and give up. The pg_xlog directory is so important that Postgres cannot function until there is enough space cleared out to start writing files again. When this problem occurs, the Postgres logs will give you a pretty clear indication of the problem. They will look similar to this:
The “PANIC” seen above is the most severe log_level Postgres has, and it basically causes a “full stop right now!”. You will note in the above snippet that a normal SQL command caused the problem, which then caused all other Postgres processes to terminate. Postgres then tried to restart itself, but immediately ran into the same problem (no disk space) and thus refused to start back up. (The “FATAL” line above was another client trying to connect while all of this was going on.)
Before we can look at how to fix things, a little background will help. When Postgres is running normally, there is a finite number of WAL files (roughly twice the value of checkpoint_segments) that exist in the pg_xlog directory. Postgres deletes older WAL files, so the total number of files never climbs too high. When something prevents Postgres from removing the older files, the number of WAL files can grow quite dramatically, culminating in the out of space condition seen above. Our solution is therefore two-fold: fix whatever is preventing the old files from being deleted, and clear out enough disk space to allow Postgres to start up again.
The first step is to determine why the WAL files are not being removed. The most common case is a failing archive_command. If this is the case, you will see archive-specific errors in your Postgres log. The usual causes are a failed network, downed remote server, or incorrect copying permissions. You might see some errors like this:
There are some other reasons why WAL would not be removed, such as failure to complete a checkpoint, but they are very rare so we will focus on archive_command. The quickest solution is to fix the underlying problem by bringing the remote server back up, fixing the permissions, etc. (To debug, try emulating the archive_command you are using with a small text file, as the postgres user. It is generally safe to ship non-WAL files to the same remote directory). If you cannot easily or quickly get your archive_command working, change it to a dummy command that always returns true:
This will allow the archive_command to complete successfully, and thus lets Postgres start removing older, unused WAL files. Note that changing the archive_command means you will need to change the archive_command back later and create fresh base backups, so do that as a last resort. Even after changing the archive_command, you cannot start the server yet, because the lack of disk space is still a problem. Here is what the logs would look like if you tried to start it up again:
At this point, you must provide Postgres a little bit of room in the partition/disk that the pg_xlog directory is in. There are four approaches to doing so: removing non-WAL files to clear space, moving the pg_xlog directory, resizing the partition it is on, and removing some of the WAL files yourself.
The easiest solution is to clear up space by removing any non-WAL files that are on the same partition. If you do not have pg_xlog on its own partition, just remove a few files (or move them to another partition) and then start Postgres. You don’t need much space—a few hundred megabytes should be more than enough.
This problem occurs often enough that I have a best practice: create a dummy file on your pg_xlog partition whose sole purpose is to get deleted after this problem occurs, and thus free up enough space to allow Postgres to start! Disk space is cheap these days, so just create a 300MB file and put it in place like so (on Linux):
This is a nice trick, because you don’t have to worry about finding a file to remove, or determine which WALs to delete—simply move or delete the file and you are done. Once things are back to normal, don’t forget to put it back in place.
The best way to get more room is to simply move your pg_xlog directory to another partition that has more space. Simply create a directory for it on the other partition, copy over all the files, then make pg_xlog a symlink to this new directory. (thanks to Bruce in the comments below)
Another way to get more space in your pg_xlog partition is to resize it. Obviously this is only an option if your OS/filesystem has been setup to allow resizing, but if it is, this is a quick and easy way to give Postgres enough space to startup again. No example code on this one, as the way to resize disks varies so much.
Checkpoint 1 1 – Verify Without Copying Permission File
The final way is to remove some older WAL files. This should be done as a last resort! It is far better to create space, as removing important WAL files can render your database unusable! If you go this route, first determine which files are safest to remove. One way to determine this is to use the pg_controldata program. Just run it with the location of your data directory as the only argument, and you should be rewarded with a screenful of arcane information. The important lines will look like this:
This second line represents the last WAL file processed, and it should be safe to remove any files older than that one. (Unfortunately, older versions of PostgreSQL will not show that line, and only the REDO location. While the canonical way to translate the location to a filename is with the pg_xlogfile_name() function, it is of little use in this situation, as it requires a live database! Thus, you may need another solution.)
Once you know which WAL file to keep by looking at the pg_controldata output, you can simply delete all WAL files older than that one. (As Craig points out in the comments below, you can use the pg_archivecleanup program in standalone mode, which will actually work all the way back to version 8.0). As with all mass deletion actions, I recommend a three-part approach. First, back everything up. This could be as simple as copying all the files in the pg_xlog directory somewhere else. Second, do a trial run. This means seeing what the deletion would do without actually deleting the files. For some commands, this means using a --dry-run or similar option, but in our example below, we can simply leave out the “-delete” argument. Third, carefully perform the actual deletion. In our example above, we could clear the old WAL files by doing:
Checkpoint 1 1 – Verify Without Copying Permission Documents
It’s worth a mention that to find files older than the specific file it’s not sufficient to just do find -not -newer, because this would actually include the file being compared against, so deleting would be disastrous for your database cluster. Be sure to include the -not -samefile in the find command. Additionally, if you have a very busy system, it’s possible that the modification timestamps on the WAL files will have the same timestamp, and so might get removed if you just blindly -delete everything. This is why it is very important to always review the output before actually deleting things.
Once you have straightened out the archive_command and cleared out some disk space, you are ready to start Postgres up. You may want to adjust your pg_hba.conf to keep everyone else out until you verify all is working. When you start Postgres, the logs will look like this:
After a few minutes, check on the pg_xlog directory, and you should see that Postgres has deleted all the extra WAL files, and the number left should be roughly twice the checkpoint_segments setting. If you adjusted pg_hba.conf, adjust it again to let clients back in. If you changed your archive_command to always return truth, remember to change it back as well as generate a new base backup
Now that the problem is fixed, how do you prevent it from happening again? First, you should use the ‘tail_n_mail’ program to monitor your Postgres log files, so that the moment the archive_command starts failing, you will receive an email and can deal with it right away. Making sure your pg_xlog partition has plenty of space is a good strategy as well, as the longer it takes to fill up, the more time you have to correct the problem before you run out of disk space.
Another way to stay on top of the problem is to get alerted when the pg_xlog directory starts filling up. Regardless of whether it is on its own partition or not, you should be using a standard tool like Nagios to alert you when the disk space starts to run low. You can also use the check_postgres program to alert you when the number of WAL files in the pg_xlog directory goes above a specified number.
In summary, things you should do now to prevent, detect, and/or mitigate the problem of running out of disk space in pg_xlog:
- Move pg_xlog to its own partition. This not only increases performance, but keeps things simple and makes things like disk resizing easier.
- Create a dummy file in the pg_xlog directory as described above. This is a placeholder file that will prevent the partition from being completely filled with WAL files when 100% disk space is reached.
- Use tail_n_mail to instantly detect archive_command failures and deal with them before they lead to a disk space error (not to mention the stale standby server problem!)
- Monitor the disk space and/or number of WAL files (via check_postgres) so that you are notified that the WALs are growing out of control. Otherwise your first notification may be when the database PANICs and shuts down!
In summary, don’t panic if you run out of space. Do the steps above, and rest assured that no data corruption or data loss has occurred. It’s not fun, but there are far worse Postgres problems to run into! :)