Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

It's mode_t; return 0 (no access) on error.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

For a number of file systems that don't have a mount point (e.g. sockfs
and pipefs), they are not marked as long term. Therefore in
mntput_no_expire, all locks in vfs_mount lock are taken instead of just
local cpu's lock to aggregate reference counts when we release
reference to file objects.  In fact, only local lock need to have been
taken to update ref counts as these file systems are in no danger of
going away until we are ready to unregister them.

The attached patch marks file systems using kern_mount without
mount point as long term.  The contentions of vfs_mount lock
is now eliminated.  Before un-registering such file system,
kern_unmount should be called to remove the long term flag and
make the mount point ready to be freed.

Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

With ib_qib options:

    options ib_qib krcvqs=1 pcie_caps=0x51 rcvhdrcnt=4096 singleport=1 ibmtu=4

a run of ib_write_bw -a yields the following:

    ------------------------------------------------------------------
     #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
     1048576   5000           2910.64            229.80
    ------------------------------------------------------------------

The top cpu use in a profile is:

    CPU: Intel Architectural Perfmon, speed 2400.15 MHz (estimated)
    Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask
    of 0x00 (No unit mask) count 1002300
    Counted LLC_MISSES events (Last level cache demand requests from this core that
    missed the LLC) with a unit mask of 0x41 (No unit mask) count 10000
    samples  %        samples  %        app name                 symbol name
    15237    29.2642  964      17.1195  ib_qib.ko                qib_7322intr
    12320    23.6618  1040     18.4692  ib_qib.ko                handle_7322_errors
    4106      7.8860  0              0  vmlinux                  vsnprintf


Analysis of the stats, profile, the code, and the annotated profile indicate:
 - All of the overflow interrupts (one per packet overflow) are
   serviced on CPU0 with no mitigation on the frequency.
 - All of the receive interrupts are being serviced by CPU0.  (That is
   the way truescale.cmds statically allocates the kctx IRQs to CPU)
 - The code is spending all of its time servicing QIB_I_C_ERROR
   RcvEgrFullErr interrupts on CPU0, starving the packet receive
   processing.
 - The decode_err routine is very inefficient, using a printf variant
   to format a "%s" and continues to loop when the errs mask has been
   cleared.
 - Both qib_7322intr and handle_7322_errors read pci registers, which
   is very inefficient.

The fix does the following:
 - Adds a tasklet to service QIB_I_C_ERROR
 - Replaces the very inefficient scnprintf() with a memcpy().  A field
   is added to qib_hwerror_msgs to save the sizeof("string") at
   compile time so that a strlen is not needed during err_decode().
 - The most frequent errors (Overflows) are serviced first to exit the
   loop as early as possible.
 - The loop now exits as soon as the errs mask is clear rather than
   fruitlessly looping through the msp array.

With this fix the performance changes to:

    ------------------------------------------------------------------
     #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
     1048576   5000           2990.64            2941.35
    ------------------------------------------------------------------

During testing of the error handling overflow patch, it was determined
that some CPU's were slower when servicing both overflow and receive
interrupts on CPU0 with different MSI interrupt vectors.

This patch adds an option (krcvq01_no_msi) to not use a dedicated MSI
interrupt for kctx's < 2 and to service them on the default interrupt.
For some CPUs, the cost of the interrupt enter/exit is more costly
than then the additional PCI read in the default handler.

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>

Add forgotten entry into the global blacklist.

Signed-off-by: Jiri Kosina <jkosina@suse.cz>

This patch drops transport_asciihex_to_binaryhex() in favor of proper
hex2bin usage from include/linux/kernel.h:hex2bin()

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Additionally this patch brings proper apply of the designator type.
However, the original code luckily has no bug, because the association
equals to 0.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: James Bottomley <jbottomley@parallels.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds the default 'Unrestricted reordering allowed' for SCSI
control mode page QUEUE ALGORITHM MODIFIER on a per se_device basis in
target_modesense_control() following spc4r23.  This includes a new
emuluate_rest_reord configfs attribute that currently (only) accepts
zero to signal 'Unrestricted reordering allowed' in control mode page
usage by the backend target device.

Reported-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@risingtidesystems.com>

This patch breaks up the ->map_task_SG() backend call into two seperate
->map_control_SG() and ->map_data_SG() in order to better address
IBLOCK and pSCSI.  IBLOCK only allocates bios for ->map_data_SG(), and
pSCSI will allocate a struct request for both cases.

This patch fixes incorrect usage of ->map_task_SG() for all se_cmd descriptors
in transport_generic_new_cmd() by moving the call into it's proper location
directly inside of transport_allocate_data_tasks()

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch contains the squashed version of forth round series cleanups
from Andy and Christoph following the post heavy lifting in the preceeding:
'Eliminate usage of struct se_mem' and 'Make all control CDBs scatter-gather'
changes.  This also includes a conversion of target core and the v3.0
mainline fabric modules (loopback and tcm_fc) to use pr_debug and the
CONFIG_DYNAMIC_DEBUG infrastructure!

These have been squashed into this third and final round for v3.1.

target: Remove ifdeffed code in t_g_process_write
target: Remove direct ramdisk code
target: Rename task_sg_num to task_sg_nents
target: Remove custom debug macros for pr_debug. Use pr_err().
target: Remove custom debug macros in mainline fabrics
target: Set WSNZ=1 in block limits VPD. Abort if WRITE_SAME sectors = 0
target: Remove transport do_se_mem_map callback
target: Further simplify transport_free_pages
target: Redo task allocation return value handling
target: Remove extra parentheses
target: change alloc_task call to take *cdb, not *cmd

(nab: Fix bogus struct file assignments in fd_do_readv and fd_do_writev)

Signed-off-by: Andy Grover <agrover@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Both backstores and fabrics use arrays of struct scatterlist to describe
data buffers. However TCM used struct se_mems, basically a linked list
of scatterlist entries. We are able to simplify the code by eliminating
this intermediate data structure and just using struct scatterlist[]
throughout.

Also, moved attachment of task to cmd out of transport_generic_get_task
and into allocate_control_task and allocate_data_tasks. The reasoning
is that it's nonintuitive that get_task should automatically add it to
the cmd's task list -- it should just return an allocated, initialized
task. That's all it should do, based on the function's name, so either the
function shouldn't do it, or the name should change to encapsulate the
entire essence of what it does.

(nab: Fix compile warnings in tcm_fc, and make transport_kmap_first_data_page
 honor sg->offset for SGLs from contigious memory with TCM_Loop, and
 fix control se_cmd descriptor memory leak)

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Since sectors is not modified, it's more straightforward to do this.

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Due to all cdbs' data buffers being referenced by scatterlists, buffers
of more than a page are not contiguous. Instead of handling this in all
control command handlers, we may be able to get away with just limiting
control cdb data buffers to one page. The only control CDBs we handle that
have potentially large data buffers are REPORT LUNS and UNMAP, so if we
didn't want to live with this limitation, they would need to be modified
to walk the pages in the data buffer's sgl.

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Previously, some control CDBs did not allocate memory in pages for their
data buffer, but just did a kmalloc. This patch makes all cdbs allocate
pages.

This has the benefit of streamlining some paths that had to behave
differently when we used two allocation methods. The downside is that
all accesses to the data buffer need to kmap it before use, and need to
handle data in page-sized chunks if more than a page is needed for a given
command's data buffer.

Finally, note that cdbs with no data buffers are handled a little
differently. Before, SCSI_NON_DATA_CDBs would not call get_mem at all
(they'd be in the final else in transport_allocate_resources) but now
these will make it into generic_get_mem, but just not allocate any
buffers.

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Implement page B1h, Block Device Characteristics, so that we can report
a medium rotation rate of 1 (non-rotating / solid state) if the
is_nonrot device attribute is set; we update the iblock backend to set
this attribute if the underlying Linux block device has its nonrot
flag set.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

The current handling of VPD page 00h (Supported VPD Pages) for INQUIRY
commands has a couple of problems:

 - The page length field is incorrectly set to 3, so the entry for 86h
   (Extended INQUIRY Data) is ignored since it is in the fourth slot.
 - Even though the code handles pages B0h and B2h, those pages aren't
   mentioned in the Supported VPD Pages list, so eg the Linux SCSI stack
   won't actually try to use them.

Fix these problems and make things more robust to avoid future problems
by moving to a table of supported VPD pages, which means that any added
VPD page support will automatically get reported on page 0.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In target_fabric_configfs_init(), we should allow fabric_mod to be NULL,
since THIS_MODULE is NULL for built-in modules.  The main method of
using the target code may be as modules, but having everything built-in
is useful eg to be able to do quick testing with "qemu -kernel".

In any case, we shouldn't bomb out fabric registration for a perfectly
valid configuration, so simply drop the check of fabric_mod.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts ft_queue_cmd() to use wake_up_process() and
ft_thread() to use schedule_timeout_interruptible(MAX_SCHEDULE_TIMEOUT)
instead of wait_event_interruptible().  This fixes a potential race with
the wait_event_interruptible() conditional with qobj->queue_cnt in
ft_thread().

This patch also drops the unnecessary set_user_nice(current, -20) in
ft_thread(), and drops extra () around two if (!(acl)) conditionals in
tfc_conf.c.

Reported-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

This patch removes the unnecessary EXTRA_CFLAGS includes, and drops the
unused -DTCM_FC_DEBUG define.

Reported-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

There is a memory leak in tcm_loop_make_scsi_hba().

If all the strstr() calls return NULL and we end up at return ERR_PTR(-EINVAL);
then we'll be leaking the memory previously allocated to tl_hba as
that variable goes out of scope.

This patch should fix the leak.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This is an off by one 'tgpt' check in tcm_loop_make_naa_tpg() that could result
in memory corruption.

Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

ibd_depth and ibd_force are used write-only.  Remove them.
ibd_major/minor can be easily retrieved from ibd_bd, so get
rid of them too.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts iblock_create_virtdevice() to use request_queue->nr_request
for se_dev_limits usage of ->hw_queue_depth and ->queue_depth for individual
struct se_device export.

It also removes the now unused defines for IBLOCK_DEVICE_QUEUE_DEPTH and
IBLOCK_MAX_DEVICE_QUEUE_DEPTH

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

transport_lookup_tmr_lun() can be called from interrupt context and
therefore needs to use IRQ-safe spinlock functions.  Fix this up, and
to make the locking work, convert the other uses of se_tmr_lock to be
IRQ-disabling.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

se_dev_check_online() is called from transport_lookup_cmd_lun(), which
as discussed before may be called from interrupt context.  So it needs
to use spin_lock_irqsave() instead of spin_lock_irq() to avoid
enabling interrupts at the wrong time.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

transport_lookup_cmd_lun() may be called from interrupt context (eg
tcm_loop_allocate_core_cmd() calls it, and it has a comment that says,
"Can be called from interrupt context"), so it needs to use
spin_lock_irqsave() instead of spin_lock_irq() to avoid enabling
interrupts at the wrong time.

(And indeed the last set of lock operations, on lun_cmd_lock, were
already using spin_lock_irqsave(), so we just need to fix the other
two locks we take)

Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds SCF_EMULATE_QUEUE_FULL support using -EAGAIN failures
via transport_handle_queue_full() to signal queue full in completion
path TFO->queue_data_in() and TFO->queue_status() callbacks.

This is done using a new se_cmd->transport_qf_callback() to handle
the following queue full exception cases within target core:

*) TRANSPORT_COMPLETE_OK (for completion path queue full)

*) TRANSPORT_COMPLETE_QF_WP (for TRANSPORT_WRITE_PENDING queue full)

*) transport_send_check_condition_and_sense() failure paths in
   transport_generic_request_failure() and transport_generic_complete_ok()

All logic is driven using se_device->qf_work_queue -> target_qf_do_work()
to to requeue outstanding se_cmd at the head of se_dev->queue_obj->qobj_list
for transport_processing_thread() execution.

Tested using tcm_qla2xxx with MAX_OUTSTANDING_COMMANDS=128 for FCP READ
to trigger the TRANSPORT_COMPLETE_OK queue full cases, and a simulated
TFO->write_pending() -EAGAIN failure to trigger TRANSPORT_COMPLETE_QF_WP.

Reported-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds a transport_handle_cdb_direct() optimization for mapping
and queueing tasks directly from within fabric processing context by calling
the newly exported transport_generic_new_cmd().  This currently expects to
be called from process context only, and will fail if called within interrupt
context.

This patch also leaves transport_generic_handle_cdb() unmodified for the
moment to function as expected with existing tcm_fc and ib_srpt fabrics,
and will be removed once these have been converted and tested with v4.1
code using transport_handle_cdb_direct().

Based on Andy's original patch here:

[PATCH 39/42] target: Call transport_new_cmd instead of adding to cmd queue

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

The release_cmd_to_pool and release_cmd_direct methods are always the same.
Merge them into a single release_cmd method, and clean up the fallout.

(nab: fix breakage in transport_generic_free_cmd() parameter build breakage
 in drivers/target/tcm_fc/tfc_cmd.c)

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch contains a squashed version to remove unused  SCF_* flags:

target: remove the unused SCF_SE_DISABLE_ONLINE_CHECK flag
target: remove the unused SCF_CMD_PASSTHROUGH_NOALLOC flag
target: remove the unused SCF_EMULATE_SYNC_UNMAP flag
target: remove the unused SCF_EMULATE_SYNC_CACHE flag

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch contains a squashed version of third round series cleanups,
improvements ,and simplfications from Andy and Christoph ahead of the
heavy lifting between round 3 -> 4 for the target core SGL conversion.

This include cleanups to the main target I/O path and other miscellaneous
updates.

target: Replace custom sg<->buf functions with lib funcs
target: Simplify sector limiting code
target: get_cdb should never return NULL
target: Simplify transport_memcpy_se_mem_read_contig
target: Use assignment rather than increment for t_task_cdbs
target: Don't pass dma_size to generic_get_mem
target: Pass sg with type scatterlist in transport_map_sg_to_mem
target: Move task_sg_num next to task_sg in struct se_task
target: inline struct se_transport_task into struct se_cmd
target: Change name & semantics of transport_get_sectors()
target: Remove unused members of se_cmd
target: Rename se_cmd.t_task_cdbs to t_task_list_num
target: Fix some spelling
target: Remove unused var from transport_generic_do_tmr
target: map_sg_to_mem: return sg_count in return value
target/pscsi: Use min_t for sector limits
target/pscsi: Unused param for pscsi_get_bio()
target: Rename get_cdb_count to allocate_tasks
target: Make transport_generic_new_cmd() available for iscsi-target
target: Remove fabric callback to allocate iovecs
target: Fix transport_generic_new_cmd WRITE comment

(hch: Use __GFP_ZERO usage for alloc_pages() usage)

Signed-off-by: Andy Grover <agrover@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch fixes the handling of WRITE_SAME_[16,32] emulation where a
WRITE_SAME_* CDB with number of blocks=0 was being rejected by SCSI
expected data transfer length overflow checking in target core.

It changes both CDB cases in transport_generic_cmd_sequencer() to use
dev->se_sub_dev->se_dev_attrib.block_size to match what sg_write_same
is sending us with --num=0.  It also fixes target_emulate_write_same()
to properly determine the num_blocks with --num=0 case to determine the
remaining range for dev->transport->do_discard().

Reported-by: Chris Greiveldinger <chris.greiveldinger@rnanetworks.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

This patch fixes a bug in the assignment of cmd->t_task.t_task_lba with
WRITE_SAME_16 to correctly use get_unaligned_be64() for the 64-bit LBA.

Reported-by: Chris Greiveldinger <chris.greiveldinger@rnanetworks.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

This patch contains the squashed version of second round of target core
cleanups and simplifications and Andy and Co.   It also contains a handful
of fixes to address bugs the original series and other minor cleanups.

Here is the condensed shortlog:

target: Remove unneeded casts to void*
target: Rename get_lun_for_{cmd,tmr} to lookup_{cmd,tmr}_lun
target: Make t_task a member of se_cmd, not a pointer
target: Handle functions returning "-2"
target: Use cmd->se_dev over cmd->se_lun->lun_se_dev
target: Embed qr in struct se_cmd
target: Replace embedded struct se_queue_req with a list_head
target: Rename list_heads that are nodes in struct se_cmd to "*_node"
target: Fold transport_device_setup_cmd() into lookup_{tmr,cmd}_lun()
target: Make t_mem_list and t_mem_list_bidi members of t_task
target: Add comment & cleanup transport_map_sg_to_mem()
target: Remove unneeded checks in transport_free_pages()

(Roland: Fix se_queue_req removal leftovers OOPs)
(nab: Fix transport_lookup_tmr_lun failure case)
(nab: Fix list_empty(&cmd->t_task.t_mem_bidi_list) inversion bugs)

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

The code in transport_add_device_to_core_hba() really intends to make sure
that neither inquiry_prod nor inquiry_rev is NULL.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

This patch contains the squashed version of a number of cleanups and
minor fixes from Andy's initial series (round 1) for target core this
past spring.  The condensed log looks like:

target: use errno values instead of returning -1 for everything
target: Rename transport_calc_sg_num to transport_init_task_sg
target: Fix leak in error path in transport_init_task_sg
target/pscsi: Remove pscsi_get_sh() usage
target: Make two runtime checks into WARN_ONs
target: Remove hba queue depth and convert to spin_lock_irq usage
target: dev->dev_status_queue_obj is unused
target: Make struct se_queue_req.cmd type struct se_cmd *
target: Remove __transport_get_qr_from_queue()
target: Rename se_dev->g_se_dev_list to se_dev_node
target: Remove struct se_global
target: Simplify scsi mib index table code
target: Make dev_queue_obj a member of se_device instead of a pointer
target: remove extraneous returns at end of void functions
target: Ensure transport_dump_vpd_ident_type returns null-terminated str
target: Function pointers don't need to use '&' to be assigned
target: Fix comment in __transport_execute_tasks()
target: Misc style cleanups
target: rename struct pr_reservation_template to pr_reservation
target: Remove #defines that just perform indirection
target: Inline transport_get_task_from_execute_queue()
target: Minor header comment fixes

Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch removes two instances of left over v3.x code performing local
scope access to struct target_core_fabric_ops->tf_subsys->su_group in
target_fabric_configfs_register() and target_fabric_configfs_deregister().

Reported-by: Christophe Fergeau <cfergeau@gmail.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>

This patch removes the now unnecessary 'unsigned char *cdb' function
parameter from transport_get_lun_for_cmd().  This also includes updating
lio-target, tcm_loop and tcm_fc usage of transport_get_lun_for_cmd().

Reported-by: Fubo Chen <fubo.chen@gmail.com>
Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org>