Features/QED/OutstandingWork: Difference between revisions

Latest revision as of 14:45, 11 October 2016

Outstanding features

Zero clusters
- Patch on mailing list [awaiting review].

Reducing consistency check time
- Reducing time window during which dirty flag is set
  - Images with backing file don't need to set dirty bit during allocating write [merged].
  - Piggyback clearing dirty bit on guest flush operation.
  - Periodic flush in order to clear dirty bit.
- Applying techniques that change the consistency check process:
  - Journaling so only the changed metadata needs to be checked for consistency.
  - Per-table dirty bits so only changed tables need to be checked for consistency.

Block device support (i.e. QED on LVM volumes)
- Solution: Keep a physical file size field in the QED header, update lazily using dirty bit logic

Live migration on shared storage
- Reopen file (open read-only first to prevent consistency check).

Encryption

Improve parallel allocating write performance
- QED currently queues allocating write requests.
- An initial per-L1/L2 table fine-grained locking approach improved performance somewhat but also regressed a benchmark.
- Needs more investigation for a good scalable solution.

In-place conversion from QCOW2

Possible future work

Discard support
- Solution: Add a freelist to the image format
- Solution: Pass discard to underlying file. Loses compact image characteristic.

Internal snapshots
- Probably not worth adding since external snapshotting is being implemented

Compression
- No strong use case for this feature

Non-technical issues

Thanks to Kevin Wolf for contributing these points:

Burden of maintaining an additional image format (basically doubles image format related work; both development and QA) [Update: Sharing code might reduce the burden, though I guess for QA purposes it would stay two independent formats.]
There are concerns that starting with a simple image format core and then adding one new thing after another on top is going to make code complexity grow very fast.
Conversion of existing images is needed to take advantage of QED.
Code review has brought up some corruption issues that are fixed by now, but have we covered everything?
- First dirty flag had to be added
- Then a bdrv_flush after COW
- What's next?
- Stefan: QED has an open specification that has been reviewed on the mailing list. There can be implementation bugs, which we can fix as soon as they are discovered. But as far as the correctness of the format, there's no better way than to get review and approval from the community and we've done that so we can be confident.

@@ Line 1: / Line 1: @@
-Thanks to Kevin Wolf for collecting these points:
+==Outstanding features==
-* Relying on file size - have we really considered all cases with respect to data integrity?
+* Zero clusters
+** Patch on mailing list '''[awaiting review]'''.
+* Reducing consistency check time
+** Reducing time window during which dirty flag is set
+*** Images with backing file don't need to set dirty bit during allocating write '''[merged]'''.
+*** Piggyback clearing dirty bit on guest flush operation.
+*** Periodic flush in order to clear dirty bit.
+** Applying techniques that change the consistency check process:
+*** Journaling so only the changed metadata needs to be checked for consistency.
+*** Per-table dirty bits so only changed tables need to be checked for consistency.
+* Block device support (i.e. QED on LVM volumes)
+** Solution: Keep a physical file size field in the QED header, update lazily using dirty bit logic
+* Live migration on shared storage
+** Reopen file (open read-only first to prevent consistency check).
+* Encryption
+* Improve parallel allocating write performance
+** QED currently queues allocating write requests.
+** An initial per-L1/L2 table fine-grained locking approach improved performance somewhat but also regressed a benchmark.
+** Needs more investigation for a good scalable solution.
+* In-place conversion from QCOW2
+==Possible future work==
+* Discard support
+** Solution: Add a freelist to the image format
+** Solution: Pass discard to underlying file.  Loses compact image characteristic.
+* Internal snapshots
+** Probably not worth adding since external snapshotting is being implemented
+* Compression
+** No strong use case for this feature
+==Non-technical issues==
+Thanks to Kevin Wolf for contributing these points:
+* Burden of maintaining an additional image format (basically doubles image format related work; both development and QA) [Update: Sharing code might reduce the burden, though I guess for QA purposes it would stay two independent formats.]
+* There are concerns that starting with a simple image format core and then adding one new thing after another on top is going to make code complexity grow very fast.
+* Conversion of existing images is needed to take advantage of QED.
 * Code review has brought up some corruption issues that are fixed by now, but have we covered everything?
 ** First dirty flag had to be added
 ** Then a bdrv_flush after COW
 ** What's next?
+** ''Stefan'': QED has an open specification that has been reviewed on the mailing list.  There can be implementation bugs, which we can fix as soon as they are discovered.  But as far as the correctness of the format, there's no better way than to get review and approval from the community and we've done that so we can be confident.
-* Planned incompatible changes must be done as soon as possible.  Checked this with Stefan, it's only zero cluster support.
+[[Category:Obsolete feature pages]]
-* After a host crash an fsck is needed. Did anyone measure the time to start up when multiple guests on multiple hosts are recovering from a system failure? File system guys consider this a severe problem.  [Update: Stefan has tested fsck time, but only with one image]
-* Migration currently doesn't work with QED, likely breaks images
-* QED doesn't work on block devices (required for some users)
-* No support for encryption, compression, internal snapshots
-* Non-technical issues:
-** Burden of maintaining an additional image format (basically doubles image format related work; both development and QA) [Update: Sharing code might reduce the burden, though I guess for QA purposes it would stay two independent formats.]
-** There are concerns that starting with a simple image format core and then adding one new thing after another on top is going to make code complexity grow very fast.
-** Conversion of existing images is needed to take advantage of QED.