- sec_register_to_crypto() and sec_unregister_from_crypto()
have been removed, the function declarations have not been
removed. Remove them.
- hisi_qm_start_qp and hisi_qm_stop_qp are called internally by the
QM. Therefore, the EXPORT_SYMBOL_GPL declaration of these
non-public interfaces is deleted.
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
1. The return value val of sec_debugfs_atomic64_get is of the
u64 type, but %lld instead of %llu is used in DEFINE_DEBUGFS_ATTRIBUTE.
Fix it.
2. In debugfs.c, since the types of q_depth and xeq_depth are u16,
the results of q_depth - 1 and xeq_depth - 1 are int rather than
u16. Use %d for int.
Signed-off-by: Zhushuai Yin <yinzhushuai@huawei.com>
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add support for the ZSTD algorithm for QAT GEN4, GEN5 and GEN6 via the
acomp API.
For GEN4 and GEN5, compression is performed in hardware using LZ4s, a
QAT-specific variant of LZ4. The compressed output is post-processed to
generate ZSTD sequences, and the ZSTD library is then used to produce
the final ZSTD stream via zstd_compress_sequences_and_literals(). Only
inputs between 8 KB and 512 KB are offloaded to the device. The minimum
size restriction will be relaxed once polling support is added. The
maximum size is limited by the use of pre-allocated per-CPU scratch
buffers. On these generations, only compression is offloaded to hardware;
decompression always falls back to software.
For GEN6, both compression and decompression are offloaded to the
accelerator, which natively supports the ZSTD algorithm. There is no
limit on the input buffer size supported. However, since GEN6 is limited
to a history size of 64 KB, decompression of frames compressed with a
larger history falls back to software.
Since GEN2 devices do not support ZSTD or LZ4s, add a mechanism that
prevents selecting GEN2 compression instances for ZSTD or LZ4s when a
GEN2 plug-in card is present on a system with an embedded GEN4, GEN5 or
GEN6 device.
In addition, modify the algorithm registration logic to allow
registering the correct implementation, i.e. LZ4s based for GEN4 and
GEN5 or native ZSTD for GEN6.
Co-developed-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace __builtin_bswap32() with swab32 in icp_qat_hw_20_comp.h to fix
the following build errors on architectures without native byte-swap
support:
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.o: in function `adf_gen4_build_decomp_block':
drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:141:(.text+0xeec): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:141:(.text+0xef8): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.o: in function `adf_gen4_build_comp_block':
drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:57:(.text+0xf64): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:57:(.text+0xf7c): undefined reference to `__bswapsi2'
Fixes: 5b14b2b307 ("crypto: qat - enable deflate for QAT GEN4")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202603290259.Ig9kDOmI-lkp@intel.com/
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In img_hash_digest(), remove the redundant return variable 'err' and
return img_hash_handle_queue() directly.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use list_first_entry_or_null() to simplify img_hash_digest() and remove
the now-unused local 'struct img_hash_dev *' variables. Use 'ctx->hdev'
when calling img_hash_handle_queue() instead of 'tctx->hdev'.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Enabling any template selects CRYPTO_MANAGER, which causes
CRYPTO_MANAGER2 to enable itself, which selects every algorithm type
option. However, pulling in all algorithm types is needed only when the
self-tests are enabled. So condition the selections accordingly.
To make this possible, also add the missing selections to various
symbols that were relying on transitive selections via CRYPTO_MANAGER.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace scatterwalk_map_and_copy() with memcpy_from_sglist() in
aspeed_ahash_dma_prepare(). The latter provides a simpler interface
without requiring a direction parameter, making the code easier to
read and less error-prone.
No functional change intended.
Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Neal Liu <neal_liu@aspeedtech.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since DES and Triple DES are obsolete, there is very little point in
maintining architecture-optimized code for them. Remove it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since DES and Triple DES are obsolete, there is very little point in
maintining architecture-optimized code for them. Remove it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since DES and Triple DES are obsolete, there is very little point in
maintining architecture-optimized code for them. Remove it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto_stdrng_get_bytes() is now always available:
- When CRYPTO_FIPS=n it is an inline function that always calls into
the always-built-in drivers/char/random.c.
- When CRYPTO_FIPS=y it is an inline function that calls into either
random.c or crypto/rng.c, depending on the value of fips_enabled.
The former is again always built-in. The latter is built-in as
well in this case, due to CRYPTO_FIPS=y.
Thus, the CRYPTO_RNG_DEFAULT symbol is no longer needed. Remove it.
This makes it so that CRYPTO_DRBG_MENU (and hence also CRYPTO_DRBG,
CRYPTO_JITTERENTROPY, and CRYPTO_LIB_SHA3) no longer gets unnecessarily
pulled into CRYPTO_FIPS=n kernels. I.e. CRYPTO_FIPS=n kernels are no
longer bloated with code that is relevant only to FIPS certifications.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently, the callers of crypto_stdrng_get_bytes() do 'select
CRYPTO_RNG_DEFAULT', which does 'select CRYPTO_DRBG_MENU'.
However, due to the change in how crypto_stdrng_get_bytes() is
implemented, CRYPTO_DRBG_MENU is now needed only when CRYPTO_FIPS.
But, 'select CRYPTO_DRBG_MENU if CRYPTO_FIPS' would cause a recursive
dependency, since CRYPTO_FIPS 'depends on CRYPTO_DRBG'.
Solve this by just making CRYPTO_FIPS depend on CRYPTO_DRBG=y (rather
than CRYPTO_DRBG i.e. CRYPTO_DRBG=y || CRYPTO_DRBG=m). The distros that
use CRYPTO_FIPS=y already set CRYPTO_DRBG=y anyway, which makes sense.
This makes the CRYPTO_RNG_DEFAULT symbol (and its corresponding
selection of CRYPTO_DRBG_MENU) unnecessary. A later commit removes it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
"stdrng" is needed only in "FIPS mode". Therefore, make
crypto_stdrng_get_bytes() delegate to either the normal Linux RNG or to
"stdrng", depending on the current mode.
This will eliminate the need to built the SP800-90A DRBG and its
dependencies into CRYPTO_FIPS=n kernels.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that crypto_default_rng, crypto_get_default_rng(), and
crypto_put_default_rng() have no users outside crypto/rng.c itself,
unexport them and make them static.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the sequence of crypto_get_default_rng(),
crypto_rng_get_bytes(), and crypto_put_default_rng() with the equivalent
helper function crypto_stdrng_get_bytes().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
All callers of crypto_get_default_rng() use the following sequence:
crypto_get_default_rng()
crypto_rng_get_bytes(crypto_default_rng, ...)
crypto_put_default_rng()
While it may have been intended that callers amortize the cost of
getting and putting the "default RNG" (i.e. "stdrng") over multiple
calls, in practice that optimization is never used. The callers just
want a function that gets random bytes from the "stdrng".
Therefore, add such a function: crypto_stdrng_get_bytes().
Importantly, this decouples the callers from the crypto_rng API. That
allows a later commit to make this function simply call
get_random_bytes_wait() unless the kernel is in "FIPS mode".
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The cpu counter used to compute the IAA device index is reset to zero
at the start of each NUMA node iteration. This causes CPUs on every
node to map starting from IAA index 0 instead of continuing from the
previous node's last index. On multi-node systems, this results in all
nodes mapping their CPUs to the same initial set of IAA devices,
leaving higher-indexed devices unused.
Move the cpu counter initialization before the for_each_node_with_cpus()
loop so that the IAA index computation accumulates correctly across all
nodes.
Fixes: 714ca27e9b ("crypto: iaa - Optimize rebalance_wq_table()")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace scnprintf() with sysfs_emit() in the three RAS error counter
sysfs show callbacks. sysfs_emit() is the recommended API for sysfs show
functions as per Documentation/filesystems/sysfs.rst; it enforces the
PAGE_SIZE limit implicitly, removing the need to pass it explicitly.
Signed-off-by: Atharv Dubey <atharvd440@gmail.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ADF_RAS_ERR_CTR_READ() expands to atomic_read(), which returns int.
The local variable 'counter' was declared as 'unsigned long', causing
a type mismatch on the assignment. The format specifier '%ld' was
consequently wrong in two ways: wrong length modifier and wrong
signedness.
Use int to match the return type of atomic_read() and update the
format specifier to '%d' accordingly.
Fixes: 532d7f6bc4 ("crypto: qat - add error counters")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
qat_comp_alg_init_tfm() acquires a compression instance via
qat_compression_get_instance_node() before calling qat_comp_build_ctx()
to initialize the compression context. If qat_comp_build_ctx() fails, the
function returns an error without releasing the compression instance,
causing a resource leak.
When qat_comp_build_ctx() fails, release the compression instance with
qat_compression_put_instance() and clear the context to avoid leaving a
stale reference to the released instance.
The issue was introduced when build_deflate_ctx() (which always returned
void) was replaced by qat_comp_build_ctx() (which can return an error)
without adding error handling for the failure path.
Fixes: cd0e7160f8 ("crypto: qat - refactor compression template logic")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Wojciech Drewek <wojciech.drewek@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace snprintf("%s", ...) with the faster and more direct strscpy().
Check if the return value is less than 0 to detect string truncation.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The get_ae_mask() function only disables individual engines based on
the fuse register, but engines are organized in clusters of 4. If the
lead engine of a cluster is fused off, the entire cluster must be
disabled.
Replace the single bitmask inversion with explicit test_bit() checks
on the lead engine of each group, disabling the full ADF_AE_GROUP
when the lead bit is set.
Signed-off-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Fixes: fcf60f4bcf ("crypto: qat - add support for 420xx devices")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The get_ae_mask() function only disables individual engines based on
the fuse register, but engines are organized in clusters of 4. If the
lead engine of a cluster is fused off, the entire cluster must be
disabled.
Replace the single bitmask inversion with explicit test_bit() checks
on the lead engine of each group, disabling the full ADF_AE_GROUP
when the lead bit is set.
Signed-off-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Fixes: 8c8268166e ("crypto: qat - add qat_4xxx driver")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Test vectors were generated starting from existing CBC(AES) test vectors
(RFC3602, NIST SP800-38A) and adding HMAC(MD5) computed with Python
script. Then, the results were double-checked on Mediatek MT7981 (safexcel)
and NXP P2020 (talitos). Both platforms pass self-tests.
Signed-off-by: Aleksander Jan Bajkowski <olek2@wp.pl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
During packet transmission, if the system is under heavy load,
the hardware might complete processing the packet and free the
request memory (req) before the transmission function finishes.
If the software subsequently accesses this req, a use-after-free
error will occur. The qp_ctx memory exists throughout the packet
sending process, so replace the req with the qp_ctx.
Fixes: f0ae287c50 ("crypto: hisilicon/sec2 - implement full backlog mode for sec")
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Many functions in cryptd.c no longer have any caller. Remove them.
Also remove several associated structs and includes. Finally, inline
cryptd_shash_desc() into its only caller, allowing it to be removed too.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Econet SoCs feature an integrated EIP93 in revision 3.0p1. It is identical
to the one used by the Airoha AN7581 and the MediaTek MT7621. Ahmed reports
that the EN7528 passes testmgr's self-tests. This driver should also work
on other little endian Econet SoCs.
CC: Ahmed Naseef <naseefkm@gmail.com>
Signed-off-by: Aleksander Jan Bajkowski <olek2@wp.pl>
Reviewed-by: Antoine Tenart <atenart@kernel.org>
Tested-by: Ahmed Naseef <naseefkm@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
AES-CCM is an AEAD algorithm supporting both encryption and
authentication of data. This patch introduces support for AES-CCM AEAD
algorithm in the DTHEv2 driver.
Signed-off-by: T Pratham <t-pratham@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
AES-GCM is an AEAD algorithm supporting both encryption and
authentication of data. This patch introduces support for AES-GCM as the
first AEAD algorithm supported by the DTHEv2 driver.
Signed-off-by: T Pratham <t-pratham@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use list_first_entry_or_null() to simplify stm32_cryp_find_dev() and
remove the now-unused local variable 'struct stm32_cryp *tmp'.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use list_first_entry_or_null() to simplify stm32_hash_find_dev() and
remove the now-unused local variable 'struct stm32_hash_dev *tmp'.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Test vectors were generated starting from existing RFC3686(CTR(AES)) test
vectors and adding HMAC(MD5) computed with software implementation.
Then, the results were double-checked on Mediatek MT7986 (safexcel).
Platform pass self-tests.
Signed-off-by: Aleksander Jan Bajkowski <olek2@wp.pl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Anti-Rollback (ARB) is a QAT GEN6 hardware feature that prevents loading
firmware with a Security Version Number (SVN) lower than an authorized
minimum. This protects against downgrade attacks by ensuring that only
firmware at or above a committed SVN can run on the acceleration device.
During firmware loading, the driver checks the SVN validation status via
a hardware CSR. If the check reports a failure, firmware authentication
is aborted. If it reports a retry status, the driver reissues the
authentication command up to a maximum number of retries.
Extend the firmware admin interface with two new messages,
ICP_QAT_FW_SVN_READ and ICP_QAT_FW_SVN_COMMIT, to query and commit the
SVN, respectively. Integrate the SVN check into the firmware
authentication path in qat_uclo.c so the driver can react to
anti-rollback status during device bring-up.
Expose SVN information to userspace via a new sysfs attribute group,
qat_svn, under the PCI device directory. The group provides read-only
attributes for the active, enforced minimum, and permanent minimum SVN
values, as well as a write-only commit attribute that allows a system
administrator to commit the currently active SVN as the new authorized
minimum.
This is based on earlier work by Ciunas Bennett.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use print_hex_dump_devel() for dumping sensitive HMAC key bytes in
hash_digest_key() to avoid leaking secrets at runtime when
CONFIG_DYNAMIC_DEBUG is enabled.
Fixes: 045e36780f ("crypto: caam - ahash hmac support")
Fixes: 3f16f6c9d6 ("crypto: caam/qi2 - add support for ahash algorithms")
Cc: stable@vger.kernel.org
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
