Skip to content
  1. Oct 20, 2020
  2. Oct 14, 2020
  3. Oct 12, 2020
  4. Oct 11, 2020
    • Linus Torvalds's avatar
      Linux 5.9 · bbf5c979
      Linus Torvalds authored
    • Jacob Keller's avatar
      scripts: remove namespace.pl · 7dfbea4c
      Jacob Keller authored
      namespace.pl is intended to help locate symbols which are defined but
      are not used externally. The goal is to avoid bloat of the namespace in
      the resulting kernel image.
      
      The script relies on object data, and only finds unused symbols for the
      configuration used to generate that object data. This results in a lot
      of false positive warnings such as symbols only used by a single
      architecture, or symbols which are used externally only under certain
      configurations.
      
      Running namespace.pl using allyesconfig, allmodconfig, and
      x86_64_defconfig yields the following results:
      
      * allmodconfig
        * 11122 unique symbol names with no external reference
        * 1194 symbols listed as multiply defined
        * 214 symbols it can't resolve
      * allyesconfig
        * 10997 unique symbol names with no external reference
        * 1194 symbols listed as multiply defined
        * 214 symbols it can't resolve
      * x86_64_defconfig
        * 5757 unique symbol names with no external reference
        * 528 symbols listed as multiply defined
        * 154 symbols it can't resolve
      
      The script also has no way to easily limit the scope of the checks to
      a given subset of the kernel, such as only checking for symbols defined
      within a module or subsystem.
      
      Discussion on public mailing lists seems to indicate that many view the
      tool output as suspect or not very useful (see discussions at [1] and
      [2] for further context).
      
      As described by Masahiro Yamada at [2], namespace.pl provides 3 types of
      checks: listing multiply defined symbols, resolving external symbols,
      and warnings about symbols with no reference.
      
      The first category of issues is easily caught by the linker as any set
      of multiply defined symbols should fail to link. The second category of
      issues is also caught by linking, as undefined symbols would cause
      issues. Even with modules, these types of issues where a module relies
      on an external symbol are caught by modpost.
      
      The remaining category of issues reported is the list of symbols with no
      external reference, and is the primary motivation of this script.
      However, it ought to be clear from the above examples that the output is
      difficult to sort through. Even allyesconfig has ~10000 entries.
      
      The current submit-checklist indicates that patches ought to go through
      namespacecheck and fix any new issues arising. But that itself presents
      problems. As described at [1], many cases of reports are due to
      configuration where a function is used externally by some configuration
      settings. Prominent maintainers appear to dislike changes modify code
      such that symbols become static based on CONFIG_* flags ([3], and [4])
      
      One possible solution is to adjust the advice and indicate that we only
      care about the output of namespacecheck on allyesconfig or allmodconfig
      builds...
      
      However, given the discussion at [2], I suspect that few people are
      actively using this tool. It doesn't have a maintainer in the
      MAINTAINERS flie, and it produces so many warnings for unused symbols
      that it is difficult to use effectively. Thus, I propose we simply
      remove it.
      
      [1] https://lore.kernel.org/netdev/20200708164812.384ae8ea@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/
      [2] https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller@intel.com/
      [3] https://lore.kernel.org/netdev/20190828.154744.2058157956381129672.davem@davemloft.net/
      [4] https://lore.kernel.org/netdev/20190827210928.576c5fef@cakuba.netronome.com/
      
      
      
      Signed-off-by: default avatarJacob Keller <jacob.e.keller@intel.com>
      Acked-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Acked-by: default avatarJakub Kicinski <kuba@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      7dfbea4c
  5. Oct 09, 2020
  6. Oct 04, 2020
  7. Sep 27, 2020
  8. Sep 24, 2020
  9. Sep 23, 2020
  10. Sep 20, 2020
  11. Sep 13, 2020
  12. Sep 07, 2020
  13. Sep 03, 2020
    • Josh Poimboeuf's avatar
      Revert "kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled" · 318af7b8
      Josh Poimboeuf authored
      
      
      Use of the new -flive-patching flag was introduced with the following
      commit:
      
        43bd3a95 ("kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled")
      
      This flag has several drawbacks:
      
      - It disables some optimizations, so it can have a negative effect on
        performance.
      
      - According to the GCC documentation it's not compatible with LTO, which
        will become a compatibility issue as LTO support gets upstreamed in
        the kernel.
      
      - It was intended to be used for source-based patch generation tooling,
        as opposed to binary-based patch generation tooling (e.g.,
        kpatch-build).  It probably should have at least been behind a
        separate config option so as not to negatively affect other livepatch
        users.
      
      - Clang doesn't have the flag, so as far as I can tell, this method of
        generating patches is incompatible with Clang, which like LTO is
        becoming more mainstream.
      
      - It breaks GCC's implicit noreturn detection for local functions.  This
        is the cause of several "unreachable instruction" objtool warnings.
      
      - The broken noreturn detection is an obvious GCC regression, but we
        haven't yet gotten GCC developers to acknowledge that, which doesn't
        inspire confidence in their willingness to keep the feature working as
        optimizations are added or changed going forward.
      
      - While there *is* a distro which relies on this flag for their distro
        livepatch module builds, there's not a publicly documented way to
        create safe livepatch modules with it.  Its use seems to be based on
        tribal knowledge.  It serves no benefit to those who don't know how to
        use it.
      
        (In fact, I believe the current livepatch documentation and samples
        are misleading and dangerous, and should be corrected.  Or at least
        amended with a disclaimer.  But I don't feel qualified to make such
        changes.)
      
      Also, we have an idea for using objtool to detect function changes,
      which could potentially obsolete the need for this flag anyway.
      
      At this point the flag has no benefits for upstream which would
      counteract the above drawbacks.  Revert it until it becomes more ready.
      
      This reverts commit 43bd3a95.
      
      Fixes: 43bd3a95 ("kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled")
      Reported-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Signed-off-by: default avatarJosh Poimboeuf <jpoimboe@redhat.com>
      Acked-by: default avatarMiroslav Benes <mbenes@suse.cz>
      Signed-off-by: default avatarPetr Mladek <pmladek@suse.com>
      Link: https://lore.kernel.org/r/696262e997359666afa053fe7d1a9fb2bb373964.1595010490.git.jpoimboe@redhat.com
      318af7b8
  14. Aug 30, 2020
  15. Aug 26, 2020
    • Nathan Huckleberry's avatar
      Makefile: Add clang-tidy and static analyzer support to makefile · 6ad7cbc0
      Nathan Huckleberry authored
      This patch adds clang-tidy and the clang static-analyzer as make
      targets. The goal of this patch is to make static analysis tools
      usable and extendable by any developer or researcher who is familiar
      with basic c++.
      
      The current static analysis tools require intimate knowledge of the
      internal workings of the static analysis. Clang-tidy and the clang
      static analyzers expose an easy to use api and allow users unfamiliar
      with clang to write new checks with relative ease.
      
      ===Clang-tidy===
      
      Clang-tidy is an easily extendable 'linter' that runs on the AST.
      Clang-tidy checks are easy to write and understand. A check consists of
      two parts, a matcher and a checker. The matcher is created using a
      domain specific language that acts on the AST
      (https://clang.llvm.org/docs/LibASTMatchersReference.html).  When AST
      nodes are found by the matcher a callback is made to the checker. The
      checker can then execute additional checks and issue warnings.
      
      Here is an example clang-tidy check to report functions that have calls
      to local_irq_disable without calls to local_irq_enable and vice-versa.
      Functions flagged with __attribute((annotation("ignore_irq_balancing")))
      are ignored for analysis. (https://reviews.llvm.org/D65828)
      
      ===Clang static analyzer===
      
      The clang static analyzer is a more powerful static analysis tool that
      uses symbolic execution to find bugs. Currently there is a check that
      looks for potential security bugs from invalid uses of kmalloc and
      kfree. There are several more general purpose checks that are useful for
      the kernel.
      
      The clang static analyzer is well documented and designed to be
      extensible.
      (https://clang-analyzer.llvm.org/checker_dev_manual.html)
      (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf
      
      )
      
      The main draw of the clang tools is how accessible they are. The clang
      documentation is very nice and these tools are built specifically to be
      easily extendable by any developer. They provide an accessible method of
      bug-finding and research to people who are not overly familiar with the
      kernel codebase.
      
      Signed-off-by: default avatarNathan Huckleberry <nhuck@google.com>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      Tested-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      Tested-by: default avatarLukas Bulwahn <lukas.bulwahn@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      6ad7cbc0
    • Masahiro Yamada's avatar
      kbuild: wire up the build rule of compile_commands.json to Makefile · 3d32285f
      Masahiro Yamada authored
      
      
      Currently, you need to manually run scripts/gen_compile_commands.py
      to create compile_commands.json. It parses all the .*.cmd files found
      under the specified directory.
      
      If you rebuild the kernel over again without 'make clean',
      .*.cmd files from older builds will create stale entries in
      compile_commands.json.
      
      This commit wires up the compile_commands.json rule to Makefile, and
      makes it parse only the .*.cmd files involved in the current build.
      
      Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order
      to the script. The objects or archives linked to vmlinux are listed in
      $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are
      listed in modules.order.
      
      You can create compile_commands.json from Make:
      
        $ make -j$(nproc) CC=clang compile_commands.json
      
      You can also build vmlinux, modules, and compile_commands.json all
      together in a single command:
      
        $ make -j$(nproc) CC=clang all compile_commands.json
      
      It works for M= builds as well. In this case, compile_commands.json
      is created in the top directory of the external module.
      
      This is convenient, but it has a drawback; the coverage of the
      compile_commands.json is reduced because only the objects linked to
      vmlinux or modules are handled. For example, the following C files are
      not included in the compile_commands.json:
      
       - Decompressor source files (arch/*/boot/)
       - VDSO source files
       - C files used to generate intermediates (e.g. kernel/bounds.c)
       - Standalone host programs
      
      I think it is fine for most developers because our main interest is
      the kernel-space code.
      
      If you want to cover all the compiled C files, please build the kernel,
      then run the script manually as you did before:
      
        $ make clean    # if you want to remove stale .cmd files [optional]
        $ make -j$(nproc) CC=clang
        $ scripts/gen_compile_commands.py
      
      Here is a note for out-of-tree builds. 'make compile_commands.json'
      works with O= option, but please notice compile_commands.json is
      created in the object tree instead of the source tree.
      
      Some people may want to have compile_commands.json in the source tree
      because Clang Tools searches for it through all parent paths of the
      first input source file.
      
      However, you cannot do this for O= builds. Kbuild should never generate
      any build artifact in the source tree when O= is given because the
      source tree might be read-only. Any write attempt to the source tree
      is monitored and the violation may be reported. See the commit log of
      8ef14c2c.
      
      So, the only possible way is to create compile_commands.json in the
      object tree, then specify '-p <build-path>' when you use clang-check,
      clang-tidy, etc.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      3d32285f
    • Masahiro Yamada's avatar
      kbuild: hide commands to run Kconfig, and show short log for syncconfig · 23cd88c9
      Masahiro Yamada authored
      
      
      Some targets (localyesconfig, localmodconfig, defconfig) hide the
      command running, but the others do not.
      
      Users know which Kconfig flavor they are running, so it is OK to hide
      the command. Add $(Q) to all commands consistently. If you want to see
      the full command running, pass V=1 from the command line.
      
      syncconfig is the exceptional case, which occurs without explicit
      command invocation by the user. Display the Kbuild-style log for it.
      The ugly bare log will go away.
      
      [Before]
      
      scripts/kconfig/conf  --syncconfig Kconfig
      
      [After]
      
        SYNC    include/config/auto.conf
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      23cd88c9
    • Sedat Dilek's avatar
      kbuild: Simplify DEBUG_INFO Kconfig handling · 695afd3d
      Sedat Dilek authored
      While playing with [1] I saw that the handling
      of CONFIG_DEBUG_INFO can be simplified.
      
      [1] https://patchwork.kernel.org/patch/11716107/
      
      
      
      Signed-off-by: default avatarSedat Dilek <sedat.dilek@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      695afd3d
  16. Aug 23, 2020
  17. Aug 18, 2020
  18. Aug 16, 2020
  19. Aug 12, 2020
  20. Aug 09, 2020
    • Masahiro Yamada's avatar
      kbuild: stop filtering out $(GCC_PLUGINS_CFLAGS) from cc-option base · 132305b3
      Masahiro Yamada authored
      
      
      Commit d26e9414 ("kbuild: no gcc-plugins during cc-option tests")
      was neeeded because scripts/Makefile.gcc-plugins was too early.
      
      This is unneeded by including scripts/Makefile.gcc-plugins last,
      and being careful to not add cc-option tests after it.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      132305b3
    • Masahiro Yamada's avatar
      kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled · e0fe0bbe
      Masahiro Yamada authored
      
      
      Currently, the top Makefile includes all of scripts/Makefile.<feature>
      even if the associated CONFIG option is disabled.
      
      Do not include unneeded Makefiles in order to slightly optimize the
      parse stage.
      
      Include $(include-y), and ignore $(include-).
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      e0fe0bbe
    • Masahiro Yamada's avatar
      kbuild: do not export LDFLAGS_vmlinux · 3ec8a5b3
      Masahiro Yamada authored
      
      
      When you clean the build tree for ARCH=arm, you may see the following
      error message from 'nm' command:
      
      $ make -j24 ARCH=arm clean
        CLEAN   arch/arm/crypto
        CLEAN   arch/arm/kernel
        CLEAN   arch/arm/mach-at91
        CLEAN   arch/arm/mach-omap2
        CLEAN   arch/arm/vdso
        CLEAN   certs
        CLEAN   lib
        CLEAN   usr
        CLEAN   net/wireless
        CLEAN   drivers/firmware/efi/libstub
      nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
      /bin/sh: 1: arithmetic expression: expecting primary: " "
        CLEAN   arch/arm/boot/compressed
        CLEAN   drivers/scsi
        CLEAN   drivers/tty/vt
        CLEAN   arch/arm/boot
        CLEAN   vmlinux.symvers modules.builtin modules.builtin.modinfo
      
      Even if you rerun the same command, the error message will not be
      shown despite vmlinux is already gone.
      
      To reproduce it, the parallel option -j is needed. Single thread
      cleaning always executes 'archclean', 'vmlinuxclean' in this order,
      so vmlinux still exists when arch/arm/boot/compressed/ is cleaned.
      
      Looking at arch/arm/boot/compressed/Makefile does not help understand
      the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are
      assigned with '=' operator, hence, they are not expanded unless used.
      Obviously, 'make clean' does not use them.
      
      In fact, the root cause exists in the top Makefile:
      
        export LDFLAGS_vmlinux
      
      Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in
      arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean
      has a command to execute. This is why the error message shows up only
      when there exist build artifacts in arch/arm/boot/compressed/.
      
      Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile
      will fix it as far as ARCH=arm is concerned, but I think the proper fix
      is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile.
      
      LDFLAGS_vmlinux in the top Makefile contains linker flags for the top
      vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for
      arch/arm/boot/compressed/vmlinux. They just happen to have the same
      variable name, but are used for different purposes. Stop shadowing
      LDFLAGS_vmlinux.
      
      This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a
      command line parameter instead of via an environment variable. LD and
      KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway,
      they must be included in cmd_link-vmlinux to allow if_changed to detect
      the changes in LD or KBUILD_LDFLAGS.
      
      The following Makefiles are not affected:
      
        arch/arm/boot/compressed/Makefile
        arch/h8300/boot/compressed/Makefile
        arch/nios2/boot/compressed/Makefile
        arch/parisc/boot/compressed/Makefile
        arch/s390/boot/compressed/Makefile
        arch/sh/boot/compressed/Makefile
        arch/sh/boot/romimage/Makefile
        arch/x86/boot/compressed/Makefile
      
      They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the
      top Makefile.
      
      We need to take a closer look at the impact to unicore32 and xtensa.
      
      arch/unicore32/boot/compressed/Makefile only uses '+=' operator for
      LDFLAGS_vmlinux. So, the decompressor previously inherited the linker
      flags from the top Makefile.
      
      However, commit 70fac51f ("unicore32 additional architecture files:
      boot process") was merged before commit 1f2bfbd0 ("kbuild: link of
      vmlinux moved to a script"). So, I rather consider this is a bug fix of
      1f2bfbd0.
      
      arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also
      considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux
      when commit 4bedea94 ("[PATCH] xtensa: Architecture support for
      Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux),
      which is now empty.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      Tested-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      3ec8a5b3
  21. Aug 02, 2020
  22. Jul 31, 2020
  23. Jul 26, 2020
  24. Jul 23, 2020
  25. Jul 19, 2020
  26. Jul 13, 2020
  27. Jul 12, 2020
  28. Jul 11, 2020
Loading