Page 5 of 7 FirstFirst ... 34567 LastLast
Results 41 to 50 of 64

Thread: Benchmarks Of ZFS-FUSE On Linux Against EXT4, Btrfs

  1. #41
    Join Date
    Apr 2010
    Posts
    271

    Default

    Quote Originally Posted by edogawaconan View Post
    If you actually know how database works, you should know that it's normal. Unless you're using crappy database, that is (mysql with myisam engine).
    I do. I would be fired within 12 hours of making backups like this at my job. Obviously you've never heard of differential/incremental/transaction log backups.

  2. #42
    Join Date
    Jul 2010
    Posts
    69

    Default

    Thanks for benchmark. I have few questions.

    Why only single disk was used for test? It is much more common to use more disk in some RAID configuration on zfs.

    Did you compiled zfs-fuse manually? Was debbuging disabled?

    What was zfs-fuse options? I'm using currently
    Code:
    DAEMON_OPTS="${DAEMON_OPTS} --fuse-attr-timeout 60.0" # caching timeout of attributes in kernel
    DAEMON_OPTS="${DAEMON_OPTS} --fuse-entry-timeout 60.0" # caching timeout of entries in kernel
    DAEMON_OPTS="${DAEMON_OPTS} --max-arc-size 1024" # maximal size of ARC in zfs-fuse
    DAEMON_OPTS="${DAEMON_OPTS} --vdev-cache-size 100" # maximal cache for dev blocks in zfs-fuse
    #DAEMON_OPTS="${DAEMON_OPTS} --disable-block-cache" # uses O_DIRECT for accesing block device, so no caching of dev blocks in kernel. seting this will disable mmap support for files
    #DAEMON_OPTS="${DAEMON_OPTS} --disable-page-cache" # disables caching of file pages in kernel
    and this still can be tuned.

    As of FUSE performance. On FUSE performance side main problem currently is xattr support, they are not cached on kernel side, and each read makes context switch to zfs-fus, to read them again, it kills performance. Be sure to have disabled xattrs and acls. For me it is most important problem as I really need xattrs.

    Beyond that you can see that READ performance if pretty good. The write performance is much worse. What of the reason (in my option) is that locking beetweeb threads is killing it (zfs-fuse is currently using about 20 threads, and if each write request will go to random one, which isn't very good idea).

    As of comparing to ext4, i also would need to ask was journal=data used? I think it should to have good comparission.


    Fuse based file system CAN be fast (see at commercial version of ntfs-3g), but still zfs-fuse is not there.

  3. #43
    Join Date
    Aug 2010
    Posts
    14

    Default

    Quote Originally Posted by locovaca View Post
    I do. I would be fired within 12 hours of making backups like this at my job. Obviously you've never heard of differential/incremental/transaction log backups.
    zfs snapshot completes in an instant. How it is done guarantees consistency of the database state (ie. no lost/duplicate log because of slight time difference in backup. There is only incomplete transaction which needs to be rolled back/forward).

    Also from postgresql documentation:

    An alternative file-system backup approach is to make a "consistent snapshot" of the data directory, if the file system supports that functionality (and you are willing to trust that it is implemented correctly). The typical procedure is to make a "frozen snapshot" of the volume containing the database, then copy the whole data directory (not just parts, see above) from the snapshot to a backup device, then release the frozen snapshot. This will work even while the database server is running. However, a backup created in this way saves the database files in a state where the database server was not properly shut down; therefore, when you start the database server on the backed-up data, it will think the previous server instance had crashed and replay the WAL log. This is not a problem, just be aware of it (and be sure to include the WAL files in your backup).

  4. #44
    Join Date
    Apr 2010
    Posts
    271

    Default

    Quote Originally Posted by edogawaconan View Post
    zfs snapshot completes in an instant. How it is done guarantees consistency of the database state (ie. no lost/duplicate log because of slight time difference in backup. There is only incomplete transaction which needs to be rolled back/forward).

    Also from postgresql documentation:
    As I said, it's the exact same as powering down your computer with the switch.

    Just because the fire extinguisher in my house should do a good job doesn't mean I burn every bag of trash in the can and put it out with the extinguisher.

    If you're going through all of the effort to set up a cron job, there's no point to ZFS snapshots for database backups. Cron an incremental backup to a remote device every hour. It's the same amount of effort, at worst the same amount of drive space, and it is 100% guaranteed to give you a sane database file.

  5. #45
    Join Date
    Aug 2010
    Posts
    14

    Default

    Quote Originally Posted by locovaca View Post
    As I said, it's the exact same as powering down your computer with the switch.
    Powering down computer with switch has risk of corrupting partially written data. It doesn't happen in zfs snapshot. How is it exactly same?

  6. #46
    Join Date
    Apr 2010
    Posts
    271

    Default

    Quote Originally Posted by edogawaconan View Post
    Powering down computer with switch has risk of corrupting partially written data. It doesn't happen in zfs snapshot. How is it exactly same?
    It is poor data management. Transaction logs shoulb be the last line of defense against failure, not the first. Timely, application specific backups should always be your first line of defense. ZFS snapshots, in this case, depend on the database engine's emergency recover processes as your only line of defense.

  7. #47
    Join Date
    Jul 2009
    Posts
    72

    Default FUSE Performance

    Quote Originally Posted by Michael View Post
    Typo, yeah, meant FUSE. Fixed. Thanks.
    ITS IN USERSPACE! If you asked for performance, your missing the point of FUSE. The point is to do things normally too unstable to support in kernel space. Doing this in userspace is of cores slow. As for getting this to near platter speed, that is impressive.

  8. #48
    Join Date
    Aug 2007
    Posts
    6,675

    Default

    ntfs-3g uses fuse too. maybe also interesting to compare.

  9. #49
    Join Date
    Aug 2010
    Posts
    14

    Default

    Quote Originally Posted by locovaca View Post
    It is poor data management. Transaction logs shoulb be the last line of defense against failure, not the first. Timely, application specific backups should always be your first line of defense. ZFS snapshots, in this case, depend on the database engine's emergency recover processes as your only line of defense.
    You went from "not a backup" to "only line of defense".
    Any database worth using should have no problem replaying transaction log.

    Also, creating zfs snapshot only takes one command and completes under a second. No need to do additional configuration and the snapshot can be sent incrementally to other machines. All this can be done without disrupting database process at all.

  10. #50
    Join Date
    Oct 2008
    Posts
    3,244

    Default

    Quote Originally Posted by edogawaconan View Post
    All this can be done without disrupting database process at all.
    As can transactional logs, so really I fail to see what this gains you in the real world and not just theoretically.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •