compFUSEd "GISMO" : new configuration

Written by Administrator Thursday, 01 May 2008 15:07

s of the release from February 2007 compFUSEd uses new format for its configuration files. Instead of using a seperate config file per mount the information can now be centralized/collected into a single file.

Unfortunately this means that you need to update you files if you used compFUSEd before. But it is not all that bad. The good news is that the different settings have not changed, only the file structure itself. In fact the file is a big concatenation of the individual settings.

The adopted format is heavily inspired by that of smb.conf (albeit much simpler). A file is split in sections, each section is associated with a mount. You must write a section for each mount now. At the end of this page you'll find an example of a configuration file (also include in the archive).

Sections

The settings are grouped per mount, a section per mount. The beginning of a section is indicated by a [...] where ... is the mount point. For example [/var/backup/]. All the following lines till the beginning of a next section or the end of file belong to the current section. Blank lines are allowed too.

ATTENTION: the name of the mount point must be identical and this include the trailing '/' to the mount point passed to compFUSEd. If not, compFUSEd will not find the required configuration! You'll get a brief error message describing the problem.

NO more defaults
You can tell compFUSEd not to compress files with a specific extension (exclude directive). 
Previously compFUSEd had a series of extension which were hard coded for exclusion. 
This is NO LONGER the case. 
You now must add for each mount a list of extension to exclude. But the format allows 
you to do that more easily too. Several line with an exclude directive are allowed. But an exclude 
directive also can take a blank space seperated list of extensions at once.
Config file location
compFUSEd GISMO now looks in 2 locations for configuration information. First in a central location, currently 
set to /usr/local/etc/compFUSEd.conf. This is the system wide config file. After parsing this file (if it exists) compFUSEd will parse
a user specific hidden file located in the user's home directory that must be called .compFUSEd

Settings specified in this file override those of the central configuration (if any). Having two sections for the same mount is possible; the last definition is always used.

Comment

Anything mentioned after a semicolon ';', a hash charcter '#' or a double forward slash '//' is considered comment till the end of that line.

; One line of comment     (smb.conf inspired)
# One line of comment     (shell script style)
// One line of comment     (C++ style)

The next line contains an exclude directive and then some comment

exclude = tgz                    // Extension of files not to compress

The backend directory

backend = /usr/cf_src

This line instructs compFUSEd to store compressed files for this mount in the /usr/cf_src directory. It is best to use an absolute path in this line.

The compression plugin

compression = /usr/local/lib/cf_lzo2.so

compFUSEd provides different compression algorithms through plugins. Specify the appropriate plugin here. You must use an absolute path.

IMPORTANT: don't change this file afterwards, compFUSEd will not be able to decompress your files!

NOTE: this is a compFUSEd provided code, not a standard library

The chunk writer plugin

writer = /usr/local/lib/writer_smarter.so

compFUSEd lets you chose between a few (currently only 2) strategies to write compressed chunks to disk. Specify which plugin you wish to use with this line.

NOTE: unlike the compression plugin the compression plugin CAN be changed between mounts

Chunk in memory per file

chunk_max = 100

compFUSEd will keep up to this number of uncompressed chunks in memory for each file. Increasing this number cause more buffering but uses more memory. Remember that this is a PER FILE setting!

Chunk size

chunk_size = 65553

Specify the size of the uncompressed chunks in bytes. Larger chunks will be better for performance but will penalize the memory usage. Don't use chunks smaller then 4096 bytes. If you do the compFUSEd overhead will be unfavourable.

Exclude

exclude = tgz                    // Extension of files not to compress
exclude = gz                    // Can have several exclude lines
exclude = bz2 tbz2 zip rar jpeg avi mp3        // Exclude several extensions at once too :)

Depending on the usage of your mount there maybe some files which you don't wish to compress. Many multimedia formats use compression. Trying to recompress these is usually a waste of time. Although compFUSEd can cope with this situation you can improve the performance be explicitly specifying which type of files (based on their extension) should not be compressed.

NOTE: extension can be in any case, compFUSEd use a case insensitive test

IMPORTANT: NO extensions are exclude by default (this was not the case previously)

Log destination

log = /var/log/cf_src.log            // You can send logging info to a file instead of stdout

When compile with debugging option compFUSEd produces quite some output. If the configuration contains no log directive the output will be sent stdout (screen). Otherwise the output is saved in the specified file.

IMPORTANT: you can NOT share a log file for different mounts!

Check on mount

force_scan                    // You can force a fsck at every mount with this

This line instructs compFUSEd to do a full sweep of all the files at mount time. This rebuild the filesystem statistics. This operation can take some time.

NOTE: compFUSEd will always perform this check if the mount was not cleanly unmounted

Example

; One line of comment (smb.conf inspired)
# One line of comment (shell script style)
// One line of comment (C++ style)

;
; The global config file is /usr/local/etc/compFUSEd.conf
; The local (as in per user) is .compFUSEd (NOTE: no .conf)
;
; IMPORTANT: the local config overrides the global one
;
;New format style
;
;[mountpoint]
;opt = value
;
; opt = backend | compression | writer | chunk_max | chunk_size | exclude
;
[/usr/src/]
backend = /usr/cf_src
compression = /usr/local/lib/cf_lzo2.so
writer = /usr/local/lib/writer_smarter.so
chunk_max = 100
chunk_size = 65553
exclude = tgz // Extension of files not to compress
exclude = gz // Can have several exclude lines
exclude = bz2 tbz2 zip rar jpeg avi mp3 // Exclude several extensions at once too :)
#log = /var/log/cf_src.log // You can send logging info to a file instead of stdout
#force_scan // You can force a fsck at every mount with this

#
# The new format centralizes the configuration for all the compFUSEd mounts
# Here another mount (used for testing!)
# This will only work if you are in the compFUSEd GISMO build dir (cfr relative paths)
#
[cf_mount/]
backend = cf_backend
compression = plugins/cf_zlib.so # Another compression plugin
writer = plugins/writer_isimple.so # Another writer plugin
chunk_max = 100 # Up to 100 chunk of 8K open per file
chunk_size = 8192 # That's 8K per chunk (uncompressed)
exclude = gz # On this mount we compress everything except .gz files


backend = /usr/cf_src
compression = /usr/local/lib/cf_lzo2.so
writer = /usr/local/lib/writer_smarter.so
chunk_max = 100
chunk_size = 65553
exclude = tgz                    // Extension of files not to compress
exclude = gz                    // Can have several exclude lines
exclude = bz2 tbz2 zip rar jpeg avi mp3        // Exclude several extensions at once too :)
#log = /var/log/cf_src.log            // You can send logging info to a file instead of stdout
#force_scan                    // You can force a fsck at every mount with this

#
# The new format centralizes the configuration for all the compFUSEd mounts
# Here another mount (used for testing!)
# This will only work if you are in the compFUSEd GISMO build dir (cfr relative paths)
#
[cf_mount/]
backend = cf_backend
compression = plugins/cf_zlib.so        # Another compression plugin
writer = plugins/writer_isimple.so        # Another writer plugin
chunk_max = 100                    # Up to 100 chunk of 8K open per file
chunk_size = 8192                # That's 8K per chunk (uncompressed)
exclude = gz                     # On this mount we compress everything except .gz files