We frequently use 'bcachefs list_journal -a' for debugging, as it
provides a record of all btree transactions, and a history of what
happened.
But it's not so useful if we immediately discard journal buckets right
after they're no longer dirty.
This tweaks journal reclaim to only discard when we're low on space,
keeping the journal mostly un-discarded.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
When we go emergency read-only, make sure we do a final write_super() to
persist counters and error counts - this can be critical for piecing
together what fsck was doing.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We often filter out EROFS errors to avoid log spew after an emergency
shutdown - journal_shutdown is just another emergency shutdown error.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This reverts
1fdbe0b184 bcachefs: Make sure c->vfs_sb is set before starting fs
switched up bch2_fs_get_tree() so that we got a superblock before
calling bch2_fs_start, so that c->vfs_sb would always be initialized
while the filesystem was active.
This turned out not to be necessary, because blk_holder_ops were
implemented using our own locking, not vfs locking.
And this had the side effect of creating a super_block and doing our
full recovery (including potentially fsck) before setting SB_BORN, which
causes things like sync calls to hang until our recovery is finished.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
wake_up() doesn't require a barrier - but wake_up_bit() does.
This only affected non x86, and primarily lead to lost wakeups after
btree node reads.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
There was a buggy version of bcachefs-tools which picked misaligned
bucket sizes when formatting, and we're also about to do dynamic block
sizes - which will allow picking logical block size or physical block
size of the device per-write, allowing for better compression ratios at
the cost of slightly worse write performance (i.e. forcing the device to
do RMW or extra buffering).
To account for this, tweak bch2_alloc_sectors_start() to properly align
open_buckets to the blocksize of the write we're about to do.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
If promote target isn't set, rebalance should still leave a cached copy
on the faster device.
Fall back to foreground_target if it's set, or allow a cached copy on
any device if neither are set.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bch2_stdio_redirect_vprintf() was missing a check for stdio->done, i.e.
exiting.
This caused the thread attempting to print to spin, and since it was
being called from the kthread ran by thread_with_stdio, the userspace
side hung as well.
Change it to return -EPIPE - i.e. writing to a pipe that's been closed.
Reported-by: Jan Solanti <jhs@psonet.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bch2_sb_disk_groups_to_cpu() goes off of the superblock member info, so
we need to set that first.
Reported-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
- Don't call bch2_trans_relock() after dir_emit(); taking a transaction
restart here will cause us to emit the same dirent to userspace twice
- Fix incorrect checking of the return value on dir_emit(): "true" means
success, keep going, but bch2_dir_emit() needs to return true when
we're finished iterating.
https://github.com/koverstreet/bcachefs/issues/867
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We can hit this limit fairly easy when we have to reconstuct large
amounts of alloc info on large filesystems.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
If loosing a btree won't cause data loss - i.e. it's an alloc btree, or
we can easily reconstruct it - we shouldn't require user action to
continue repair.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Restart handling in the previous patch was incorrect, so: move btree
operations into a separate helper, and run it with a lockrestart_do().
Additionally, clarify whether pagecache or the btree takes precedence.
Right now, the btree takes precedence: this is incorrect, but it's
needed to pass fstests. Add a giant comment explaining why.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs currently populates fiemap data from the extents btree.
This works correctly when the fiemap sync flag is provided, but if
not, it skips all delalloc extents that have not yet been flushed.
This is because delalloc extents from buffered writes are first
stored as reservation in the pagecache, and only become resident in
the extents btree after writeback completes.
Update the fiemap implementation to process holes between extents by
scanning pagecache for data, via seek data/hole. If a valid data
range is found over a hole in the extent btree, fake up an extent
key and flag the extent as delalloc for reporting to userspace.
Note that this does not necessarily change behavior for the case
where there is dirty pagecache over already written extents, where
when in COW mode, writeback will allocate new blocks for the
underlying ranges. The existing behavior is consistent with btrfs
and it is recommended to use the sync flag for the most up to date
extent state from fiemap.
Signed-off-by: Brian Foster <bfoster@redhat.com>
The bulk of the loop in bch2_fiemap() involves processing the
current extent key from the iter, including following indirections
and trimming the extent size and such. This patch makes a few
changes to reduce the size of the loop and facilitate future changes
to support delalloc extents.
Define a new bch_fiemap_extent structure to wrap the bkey buffer
that holds the extent key to report to userspace along with
associated fiemap flags. Update bch2_fill_extent() to take the
bch_fiemap_extent as a param instead of the individual fields.
Finally, lift the bulk of the extent processing into a
bch2_fiemap_extent() helper that takes the current key and formats
the bch_fiemap_extent appropriately for the fill function.
No functional changes intended by this patch.
Signed-off-by: Brian Foster <bfoster@redhat.com>
FIEMAP_FLAG_SYNC handling was deliberately moved into core code in
commit 45dd052e67 ("fs: handle FIEMAP_FLAG_SYNC in fiemap_prep"),
released in kernel v5.8. Update bcachefs accordingly.
Signed-off-by: Brian Foster <bfoster@redhat.com>
At the end of the inode, on an extents iterator, peek_slot() has to
advance to the next position to avoid returning a 0 size extent, which
is not allowed.
Changing iter->pos confuses peek_prev(), but we don't need to call
peek_slot() in this case.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
The issue this assert is guarding against is that in
BTREE_ITER_filter_snapshots mode we only want to be iterating within a
single inode number - if we iterate into another inode number with keys
for a different snapshot tree, we'll loop arbitrarily long before
finding a key we can return.
This comes up in the unit tests, where we're using inode 0 for our test
keys.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
If we aren't mounting with the correct degraded option, it's helpful to
know that before we fail to mount degraded.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
casefolding results in additional aliases on lookup for the
non-casefolded names - these need invalidating on unlink.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Add casefolding to bch2_lookup_trans:
During the delay between when casefolding was written and when it was
merged, the main filesystem lookup path grew self healing - which meant
it was no longer using bch2_dirent_lookup_trans(), where casefolding on
lookups happens.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We had a buggy release of bcachefs-tools that wasn't properly aligning
bucket sizes.
We can't ask users to reformat - and it's easy to teach the allocator to
make sure writes are properly aligned.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Previously, copygc and rebalance weren't started until the very end of
mounting, after all recvoery passes have finished.
But copygc really should be started earlier, since it may be needed for
allocations to make forward progress. Additionally, we've been seeing
occasional bug reports where starting the kthread fails due to a pending
signal - i.e. we're getting timed out by systemd (during a version
upgrade), but we're not seeing the signal until mount is about to
complete.
Additionally, we now have copygc/rebalance explicitly wait for
check_snapshots to complete (if being run); they require that for
snapshot_is_ancestor() in the data move path.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Don't use a continue; this simplifies the next patch where
run_recovery_passes() will be responsible for waking up copygc and
rebalance at the appropriate time.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
