bch2_journal_key_insert() used to assume that the key passed to it was
allocated with kmalloc(), and on success took ownership. This patch
deletes that behaviour, making it more similar to
bch2_trans_update()/bch2_trans_commit().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
If the allocator threads start before journal replay has finished
replaying alloc keys, journal replay might overwrite the allocator's
btree updates.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Now that bch2_bucket_alloc_new_fs() isn't looking at bucket marks to
decide what buckets are eligible to allocate, we can clean up the
filesystem initialization and device add paths. Previously, we had to
use ancient code to mark superblock/journal buckets in the in memory
bucket marks as we allocated them, and then zero that out and re-do that
marking using the newer transational bucket mark paths. Now, we can
simply delete the in-memory bucket marking.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This changes bch2_bucket_alloc_new_fs() to a simple bump allocator that
doesn't need to use the in memory bucket array, part of a larger patch
series to entirely get rid of the in memory bucket array, except for
gc/fsck.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Similarly to bch2_btree_delete_range_trans(), bch2_inode_delete_keys()
may sometimes split compressed extents, and needs to pass in a disk
reservation.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
It'll now be handled at format time and in sysfs like other options - it
still can only be set at format time, though.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This adds flags for options that must be a power of two (block size and
btree node size), and options that are stored in the superblock as a
power of two (encoded extent max).
Also: options are now stored in memory in the same units they're
displayed in (bytes): we now convert when getting and setting from the
superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
It wasn't updated for snapshots - it's iterating across keys in all
snapshots, so needs to be specifying BTREE_ITER_ALL_SNAPSHOTS.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This adds a flag to not mark the initial btree_path as preserve, for
paths that we expect to be cheap to reconstitute if necessary - this
solves a btree_path overflow caused by need_whiteout_for_snapshot().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This fixes some bugs when we hit an error very early in the filesystem
startup path, before most things have been initialized.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This puts a load behind some branches before where it's used, so that it
can execute in parallel with other loads.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
The main function of bch2_sort_repack_merge() was to call .key_normalize
on every key, which drops stale (cached) pointers - it hasn't actually
merged extents in quite some time.
But bch2_gc_gens() now works on individual keys - we used to gc old gens
by rewriting entire btree nodes. With that gone, there's no need for
internal btree code to be calling .key_normalize anymore.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This puts the btree_transactions sysfs/debugfs file behind a separate
config option - it's highly useful, but not cheap enough to enable
permenantly.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We recently added an assertion that when we truncate a file to 0,
i_blocks should also go to 0 - but that's not necessarily true if we're
doing an emergency shutdown, lots of invariants no longer hold true in
that case.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This fixes some compiler warnings that only trigger in userspace - dead
code, a maybe uninitialed variable, a maybe null ptr passed to printk.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This moves some common code into alloc_mem_to_key(), which translates
from the in-memory format for a bucket to the btree key format.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This adds a new helper that much like the one we have for inode updates,
that allocates the packed alloc key, packs it and calls
bch2_trans_update.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This adds more latency/event measurements and breaks some apart into
more events. Journal writes are broken apart into flush writes and
noflush writes, btree compactions are broken out from btree splits,
btree mergers are added, as well as btree_interior_updates - foreground
and total.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
bch2_btree_delete_range() can split compressed extents, thus needs to
pass in a disk reservation when we're operating on extents btrees.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
It doesn't make much sense to be erasure coding cached pointers, we
should be erasure coding one of the dirty pointers in an extent. This
patch makes sure we're passing BCH_WRITE_CACHED when we expect the new
pointer to be a cached pointer, and tweaks the write path to not
allocate from a stripe when BCH_WRITE_CACHED is set - and fixes an
assertion we were hitting in the ec path where when adding the stripe to
an extent and deleting the other pointers the pointer to the stripe
didn't exist (because dropping all dirty pointers from an extent turns
it into a KEY_TYPE_error key).
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
We have two radix trees of stripes - one that mirrors some information
from the stripes btree in normal operation, and another that GC uses to
recalculate block usage counts.
The normal one is now only used for finding partially empty stripes in
order to reuse them - the normal stripes radix tree and the GC stripes
radix tree are used significantly differently, so this patch splits them
into separate types.
In an upcoming patch we'll be replacing c->stripes with a btree that
indexes stripes by the order we want to reuse them.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
A user reported hitting this assertion, and we can't reproduce it yet,
but it shouldn't be fatal - so convert it to a warning.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
With snapshots, bch2_trans_update() has to check if we need a whitout,
which can cause a transaction restart, so this is important now.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Switch to one line of output per pr_buf() call - longer lines but quite
a bit more readable.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
We've got three types of options now - filesystem, device and inode, and
a given option may belong to more than one of those types.
This patch changes the options to specify explicitly when they're a
filesystem option - in the future we'll probably be adding more device
options.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
With erasure coding, copygc's count of sectors to move was off, which
matters for the debug statement it prints out when it's not able to move
all the data it tried to.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This converts journal_write_delay, journal_flush_disabled, and
journal_reclaim_delay to normal filesystems options, and also adds them
to the superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
When we added the stripe and stripe_redundancy fields to alloc keys, we
neglected to add them to the functions that convert back and forth with
the in-memory types.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This simplifies the code quite a bit and eliminates an inconsistency - a
given bkey doesn't necessarily translate to a single replicas entry for
disk space accounting.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This changes the bch2_mark_key() and related paths to take mark lock
where it is needed, instead of taking it in the upper transaction commit
path - by pushing down locking we'll be able to handle fsck errors
locally instead of requiring a separate check in the btree_gc code for
replicas being marked.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This changes bch2_trans_fs_usage_apply() to handle failure (replicas
entry missing) by reverting the changes it made - meaning we can make
the main transaction commit path a bit slimmer, and perhaps also
simplify some locking in upcoming patches.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
make_extent_indirect() was missing the
BTREE_UPDATE_INTERNAL_SNAPSHOT_NODE - it's updating the extent in the
original snapshot, not the curret one.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Start a new header, errcode.h, for bcachefs-private error codes - more
error codes will be converted later.
This patch just converts bucket_alloc_ret so that they can be mixed with
standard error codes and passed as ERR_PTR errors - the ec.c code was
doing this already, but incorrectly.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This tweaks the fallocate code to also update the page cache to reflect
the new on disk reservations, giving us better i_sectors consistency.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This patch adds code to read page state before writing to pages that
aren't uptodate, which corrects i_sectors being tempororarily too large
and means we may not need to get a disk reservation.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
# Conflicts:
# fs/bcachefs/fs-io.c
Reading from cached data, which calls bch2_bucket_io_time_reset(), is
leading to transaction iterator overflows - this standardizes the
workaround.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Quota support was disabled when snapshots were released, because of some
tricky interactions with snpashots. We're sidestepping that for now -
we're simply disabling quota accounting on snapshot subvolumes.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
This fixes another i_sectors accounting bug - we need to differentiate
between dirty writes that overwrite a reservation and dirty writes to
unallocated space - dirty writes to unallocated space increase
i_sectors, dirty writes over a reservation do not.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
When bch2_truncate_page() discards dirty sectors in the page cache, we
need to account for that - we don't need to account for allocated
sectors because that'll be done by the bch2_fpunch() call when it
updates the btree.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
We were setting BCH_FS_ERROR on startup if the superblock was marked as
containing errors, which is not what we wanted - BCH_FS_ERROR indicates
whether errors have been found, so that after a successful fsck we're
able to clear the error bit in the superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
